Thursday August 17, 2006

There's an interesting set of articles on computational photography in the August edition of IEEE Computer. While I hadn't heard the term before, the concept makes sense. To date, digital camera technology has been primarily focused on replicating the capabilities of the film cameras they are rapidly replacing. Computational photography looks at ways the photographic experience might be extended by taking advantage of onboard processing, new sensor technologies, new lens, etc.

To make this more concrete, one example illustrated in Computer uses a standard digital camera with a conical mirror mounted in front of the lens to provide an enhanced capability. A photograph using this arrangement has two areas: the central area containing the direct image of the subject that was photographed, and a surrounding annular region that contains a very distorted view of the subject as reflected on the conical mirror. In fact, there are two views of the subject in the mirror and these views can be used to compute the 3D geometry of the subject and create a real 3D representation using offline processing.

As I read the articles, I realized that industry has already started moving towards this vision of a future photography. I recently bought a new compact camera for travel that has two interesting features that are modest moves in the direction highlighted in the Computer articles.

The first feature is auto skew correction that is computed locally within the camera. In this mode, the camera examines the just-snapped photo for trapezoidal areas that represent skewed rectangles. It will identify up to five such regions and then allow the user to choose which one to skew correct. The camera then proceeds to unwarp that section of the image and map it to a rectangular area, presumably restoring the object to its correct shape. Initially, I didn't think I'd have much use for these seemingly esoteric feature, but it has turned out to be very useful. The example below shows a shot of a projected microfilm image (a passenger arrival list) at NARA. The original shot is on the left and the skew corrected version is on the right. Again, the cool thing is that this operation is done entirely within the camera, which saves both the original shot and the skew corrected version.

The other feature in my camera relates to a concept called The Moment Camera, which is also described in IEEE Computer. The basic idea is to capture a small slab of space-time with the camera rather than a single instant and then use this additional information in various ways to enhance the end result. For example, multiple exposures could be taken at different focal distances and then combined to create a single image with enhanced depth of field. Or one could simply peruse the slab and select the instant that best captures the moment you were trying to capture. The article contains the interesting observation that about 10% of our waking life is spent with our eyes closed, hence all those droopy-lidded shots of the family that we all hate so much.

Back to my camera. It has a continuous mode that allows it to capture several exposures in rapid succession. Nothing unusual there. But it supports two additional continuous modes that remind me more of this space-time slab concept. In the first mode, a single press of the shutter button creates a single image file containing 16 shots, each shot taken once every 1/7.5 seconds. The second mode is more aligned with the moment camera idea. Hold down the shutter button and the camera will continuously buffer the last 16 shots and then store the latest 16 when you release the shutter. Analyze that golf swing, catch the moment.

These are all baby steps on the way to a new era of digital photography. It will be interesting to see what new capabilities become available and how they'll be used.

For the curious, my new camera is a Ricoh Caplio R4. I'll cover it in detail in a future post, but for now suffice to say I'm quite happy with it.