Misc. May: Intel Trigate, Microscopes and GE printing

Its been such a busy May for me since it seems like the news from the micro tech world keeps coming at a faster and faster pace. There are tons of things I'd like to discuss in detail, but instead I'm just listing it here:

a) The big news this month, of course, is that Intel is building "3D" gates that have a vertical fin to allow closer packing. PCmag has a good overview of the technology and the Economist explains the corporate drama surrounding the Intel-ARM war to own the future of micro device processors and the implications of the Trigate technology in that battle. The bottom line is that Trigate will allow low power devices that will extend Moore's law further into the decade. (Not for too long: just 2 years. That gives you an idea of how tough it will be to make hardware for micro machines in the future).

b) RK sent me a link about IBM fellows and Nobel prize winners discussing their scanning tunneling microscope. Now I don't know much about quantum effects, but I always thought they were esoteric theories that were used in nuclear power plants and so on. But the STM is essentially a camera that uses quantum theory to take a picture. Its fascinating stuff, and I decided to look at the history of microscopes in wikipedia, which has a good summary. You should really spend an afternoon and learn about microscopes (as I did), but here are two interesting facts to pique your interest and get you started:

• The use of optical microscopes took off after a heuristic was discovered called the Abbe Sine Law
• Electron microscopes are quite old. The transmission electron microscope was invented in the 30s and the scanning electron microscope in the 60s.
  

c) The BBC has a cool video about Near Field Communication technology that is converting our phones into e-wallets (not just credit cards, but ID cards and driving licenses too).

d) Here is a useful image sensor blog link by the founder of Advasense. Its a good place to go and search for topics on image sensor hardware, say back-illuminated sensors or whatever it is you are currently thinking about.

e) GE is going to be the first big manufacturing company that will produce products (ultrasound devices) by printing them. (Article from the Economist).

New panoramic tech plus a note on 3D printing

Its amazing how web-based panoramic image browsing has become such an accepted and normal part of our life: think how accessible Google Streetview, Microsoft's Photosynth and Gigapan are. And this technology hasn't finished changing our world yet: its moving into two new territories.

The first is "indoor streetview" that Google is starting up for businesses. Thanks to RK for bringing this to my attention. Soon you'll be able to invite Google over and get a service setup so that potential customers can browse through images of your small business. Since Google Streetview pushed Google into automated cars (see this link), maybe iRobot needs to start worrying about competition from Google indoor robots? Why stop at businesses, when people already use web-cams to monitor small children from work. Looks like a possible direction for small, portable, low-power visual sensors.

The second direction that panaromic technology is moving toward is stereoscopy. You can find projects to create stereoscopic gigapans and TONS of stereo panoramas on the web. However, these don't actually "work" when you zoom in to see details. Its non-trivial to model the geometry of the scene and re-render 3D images with the right disparity as you closely look at small objects in the scene. MC pointed out that the "right" way to do this would be to capture the light-field at the viewing location. To do this, you would need to smoothly move a video camera in an arc, instead of a stereo pair of still cameras.

Side note on 3D printing: AJ has been working on 3D printer kits that can be assembled for classrooms. Related tools have inspired new kinds of art, as the Nytimes explains.

How long before we have printers that can print printers? The singularity approacheth....

Kinect

Yesterday I took the train down to Brown see a set of cool vision talks. One of the presenters talked about "RGB-D" images, which mean different things to different people. To me, RGB-D is an image abstraction. It augments the red, green and blue color channels with a fourth channel which is the pixel depth (the "D" in RGB-D).

Although there are tons and tons and tons of methods that reconstruct scene depths from images, RGB-D folks are agnostic to these. They have an interesting abstraction subtext to their work, which could be summarized as: "Pick a vision reconstruction method that works really well. Lets have it implemented as a hardware black box . Lets now celebrate the fact that we don't have to worry about depth-reconstruction/shape-from-X again. We'll assume that we have perfect depth, and lets build some cool vision on top of that."

I think this attitude is awesome since it gets people to think beyond scene reconstruction.

One of the "black boxes" that give RGB-D images is the Microsoft Kinect. Whats interesting for micro computer vision is an interesting sub-component of the Kinect: a tiny IR projector.

Correction: The Kinect has no IR projector. See updates below.

You can see pics of it here. The Kinect has a stereo-pair-with-projector system. The projector adds texture for the stereo system, but its "projected light" is unnoticed since its in IR and so invisible (You can see the projected pattern here, where a video was taken with a night-vision camera.)

I believe the projector pattern is fixed. This means Microsoft could have gotten away with a very bright IR LED and a physical, printed texture pattern. Why did they use a projector? I'm not sure, but there is an opportunity to hack and control the projector. I'm surprised to have not found anything along those lines yet on the web. I'm particularly curious about the projector's frame rate, and whether high-speed applications are possible.

Update: We have confirmation that the Kinect does not have a projector at all. Thanks to AV for the update. (Also, people actually read the blog.)

Update 2: Thanks to GD for pointing out this website kinecthacks.net

Stereoscope for the iPhone

The stereoscope was one the first 3D viewing devices. Its a low-tech device that basically places an opaque occluder between your eyes, forcing a separation between what your left and right eye can see. A stereo pair placed correctly on either side of the occluder gives the viewer an impression of depth.

Hasbro has created a very interesting update to the stereoscope for the 21st century. A binoculars-like device pretty much plays the role of an occluder. At the end of the "binoculars", an iPhone or smartphone is attached. A Hasbro app formats 3D content appropriately, so that each eye sees only the correct image through the "binocular" tubes.

This was one of those "why didn't I think of that!" moments for me. Very cool stuff from Hasbro. However, one can't help but think the way forward for mobile 3D devices is more of the autostereoscopic optics from the likes of Nintendo, with its 3DS. The Hasbro device is useful for people who already have an iPhone or similar device and who don't mind the device form factor.

New Micro Four Thirds models

The Micro four thirds cameras were launched by Olympus and Panasonic about three years ago. A quick recap: SLR cameras have large sensors and pixels and therefore low noise. They also have a "mirror in the loop" that allows the viewfinder to show *exactly* what is going to be photographed. This is done by beamsplitting the incoming light into two paths, one of which goes to the sensor and the other to the viewfinder. These two features force SLR cameras to have a large form factor.

The Micro four thirds system basically tries to be a "pocket SLR" by reducing the sensor size (but not by much) and removing the mirror in the viewing path. It also tries to give the consumer an "SLR feel" by having removable and interchangeable lenses. The nytimes has a nice article about the new cameras, and the sizes are 4.4x2.7x1.3 inches.

This blog always likes to see if innovations meant for the marketplace have an impact on research. I'm not seeing any particular feature of these cameras that we could use in research, since neither the small sizes nor the high quality of images are a game-changer. However, you can think of these cameras as a poor man's SLR. Perhaps we could exploit the cheapness factor in some areas of appearance capture, where many cameras are used...are there applications that need 100s of SLR-like cameras?

A note about image noise and sensor resolution. Large pixel sizes collect more light, so its obvious that this would increase SNR. However, when you purchase a camera, you rarely find two candidate cameras with the same resolution, but one with larger pixel sizes: that would be an easy choice. Instead, you may find one camera with a large sensor and small pixels, while another has a smaller sensor but with larger pixels. Each individual pixel in the second camera should have higher SNR, but the overall sensor size of the first one is larger, and that does have some benefit.

There are sites that explain these relationships in detail. I just want to point out that the right decision is not obvious since resolution and SNR get mixed up when you compare across both those quantities.

ICCP post 2

Yesterday was a packed day at ICCP 2011. Illah Nourbakhsh started out the day with an amazing talk on Gigapan imaging. The philosophical breakthrough he made was to contrast a gigapixel image to a real image.

If you view a gigapixel image you cannot see the "whole" picture, since the resolution is huge (unlike a snap of a family picnic). You have to zoom in and explore the image. Now imagine the image is of some place you haven't ever been to, lets say Mars. So now when you zoom in, Illah says that you are exploring that place, not just viewing a picture.

The optics session that followed was pretty amazing too. Oliver Cossairt had a great talk on the new gigapixel camera from Shree Nayar's lab. The final presentation in this session was Hand-Held Schlieren Photography with Light Field Probes, which won the best paper award at ICCP.

ICCP post 1

I'm sitting in the beautiful Rashid Auditorium listening to the opening of ICCP 2011!

Yesterday there were some tutorials.

I loved the talk by Peter Belhumeur on digital plant detection. The topic may sound a little esoteric to the general audience, but the results were beautiful and the paper was interesting. A nytimes article is here.