A very high resolution binary sensor

I've been reading this article from EPFL, about a new kind of imaging sensor. Each pixel, or photodetector, measures the light by producing a binary value of either 0 or 1. So, each pixel is, loosely speaking, a photon counter. The advantage of this sensor is that it has the potential to be really fast (high frame rate). Once such binary data is collected, a grayscale image can be obtained from it by applying image processing techniques. This is similar to what the eye does when it sees the spatial dithering in newspaper halftones.

RIP Bryce Bayer

Kodak researcher Bryce Bayer, who created the widely used Bayer filter, has died.

Digital microscope

I randomly came across digital microscopes, which are basically single lens devices that use the high resolution of the sensor (plus a large monitor) to give the impression of magnification. The concept is basically macro photography packaged as a kind of non-optical magnification. You can buy one for about $100. While I was initially skeptical, I learned about some of the challenges faced when building these devices. These include matching the single lens throughput with the sensor resolution and dealing with issues relating to lighting and frame rates. Here is a site with more details including some images of small objects. Some interesting trends include a publication that uses a second lens (an eyepiece) with a cellphone/tablet to enable low-cost lab applications. I also found an application where a company built a stereoscopic digital microscope and hooked it up to a stereoscopic TV. So you basically can see live 3D images of small objects, like an insect.

Antenna lens

The Economist has an interesting article about a new kind of lens made of tiny antennae. When light hits the antennae, the photos are absorbed before being re-emitted (retransmitted?) in a converging direction.

Broken push-broom

Thanks to E for this link, where it looks like an artist bought an expensive push-broom camera and broke its motor. The camera then keeps taking pictures along the same column. Stringing these together gives an interesting image: a static pushbroom projection of a moving world.

A cardinality sensor

Recently I was talking to Takeo Kanade, and he mentioned an old paper of his with Vladimir Brajovic, where the image produced by the sensor is sorted according to radiance. If two pixels receive the same light energy, then they get the same rank. Now you might think this is uninteresting, since you could just take a regular image and sort it in software. But remember, I said sorted by radiance (which is continuous), not intensity measurement (which is usually 0 - 255). For a scene that is over-exposed, with a regular sensor, you'd get an image that is mostly 255s. With the cardinality sensor, you'd get an image with lots of information, since it would preserve the rank of each scene point (which depends on the number of other scene points that are darker or brighter). Essentially each pixel is in a race to be overexposed, and once the pixel reaches the value 255, it increases a counter variable by 1 and takes the counter variable's value (which is its rank). Ties get the same value, but increase the counter variable by the number of tied pixels.

Random articles

Here are some articles I have been reading recently:

The latest update in the CMOS vs. CCD war.

What kind of sensors go on a Mars robot?

Hoping to print tons of micro-sensors in your garage? We are getting there. 

Sensors

I had a discussion with AJ about sensors and then went to the Wikipedia page. The description includes a nice definition (a sensor is a device which receives and responds to a signal) but does not elegantly describe the types of sensors (just a laundry list of different devices).

I think there are only two classes of sensors:

1) Contact or Force sensors, which I call touch sensors.

2) Electromagnetic sensors, which I loosely call light sensors.

All sensors fall into these categories (see postscripts):

Touch sensors: Anything I hear from you is basically your voice-box pushing a bunch of air to hit my ear drum. This is just like when you sit in a jacuzzi and feel the water jets. You'd say the jets seem like they are pushing against you; that is exactly what sound is doing to our ears. Time-of-flight sonars work in a similar way. Chemical sensors and taste sensors (like your tongue) are sensing bits of the object scattered in the medium. These fragments actually push or scrape against receptors. Gravitational and magnetic sensors (like a compass), again, are due to a force being applied directly onto your sensor.

Electromagnetic sensors: These, like photodetectors or heat/temperature sensors, generate a signal because different types of radiation influence the sensor. In this sense (pun intended) light sensors are transducers: they just convert electromagnetic energy falling on them into an electric reading.

Postscript 1: So whats an image? An image is just a matrix of values that store the responses of one or more sensors. You can have an "image" of sonar sensors or of a group of chemical sensors. If the sensor is on-off in response, the image is binary. The powerful concepts of images and video have nothing particularly to do with light.

If you keep running with this idea, then you'll realize that the retina is not the only thing generating an image in your head. Your tongue is also an array of sensors and it produces an "image" of the food you are eating. Actually its creates a video, because it measures images of taste responses over time. You can't display this video, because your eyes wouldn't make sense of it. But what your optic nerve cannot understand your chorda tympani (your taste nerve) absolutely can.

Postscript 2: Where do gyroscopes fit in? Gyros don't measure anything - they stay the same despite what is happening around them. If you are philosophical, you might say they measure inertia. I put them in the category of touch sensors as a sort of "null set" of that class because they resist all forces and contact.

Postscript 3: Yes, what are magnets really? There is a current loop definition which says that magnets are electric currents that circle around in an object, creating a magnetic field. But in the end, even if the source of the force is electromagnetic, your sensor detects the field by being pulled by it. I'd call that a touch sensor.

Code resource

Here is a good resource for code for optimization problems in vision that show up a lot while making small sensors. Thanks to AJ for the link.

Graphene sensors

The Economist has a fascinating article about how researchers are using graphene, interspersed with tiny dots of lead sulphide, to absorb light and produce electrons that can be converted into a signal. The cool thing about these proposed photodetectors is that they might be flexible, allowing many applications. They possibly could also be made very small and might even allow the creation of a "light transistor". 

Stereoscopy on small devices

Thanks to EJ for sending me this article. Its a nice overview of the challenges faced when showing stereoscopic content meant for one screen on another, different display. The most interesting section, for me, was about 3D content on mobile phones. The author points out two interesting things: (1) Given the size of the screen, 3D movies and games for portable devices are *usually* easy on the eyes (the disparity-focus headache factor is low). However: (2) The "miniaturization" effect (where objects appear toy-like) is very strong. This means that (outside of "cute'" games) most scenes will appear (to paraphrase the author) as if strangely in a small box. I think the field is wide open for formats and representations that will allow good viewing of 3D content on small devices.

The 21st century's foundry

I've been busy with a lot of moving stuff, so that is why there has been a lack of posts. Hopefully I'll write more in the next few months.

I've been really enjoying (and nodding while reading) the Economist's special report on personal manufacturing as the next industrial revolution.

Flashing signs

The Economist has a neat article about how Casio is getting smartphones to talk to each other using flickers in their displays. Pulses of light are the way DMD and MicroVision projectors send out light, so you might be able to extend this idea to portable projectors.

Eye dust in the sky

The Economist has an article about civilian micro air vehicles and how society might react to them. Two interesting technical things I learned from the article was that (1) AeroVironment makes most of the US's micro air vehicles and (2) A company called SARA makes acoustic sensors for micro-air vehicles that block out wind noise and can detect other planes and do collision avoidance.

I want to talk about the article's focus on the societal impact of tiny visual sensors. The advent of such small camera-enabled robots will be the first of many technologies, with serious privacy concerns, that will probably come out in the next 10-20 years. For now, the available micro air vehicles are relatively large (laptop size or smaller). But smart dust and other futuristic visions are closer than you think. Sure, these ideas have been thrown around for the past two decades. But as we get close to the point when science fiction becomes commercial product, the human impact will be substantial and we must prepare for it. Millions of costless micro-cameras will envelope our lives in a 24 hour obtrusive sensor blanket. Combined with face detection, these micro-sensors will be able to identify individuals over a large area quite quickly; frankly, this would mean that end of privacy as we know it. Of course there will be air filters and particle blockers (for some predictions, see the Diamond Age), but the relentlessness of manufacturing processes and the possibility that these small devices will exhibit self-replication would probably overwhelm any stop-gap measure. I'm not sure I have any comments on how to make this situation better, because I'm pretty sure its inevitable.