Computational Camera and Programmable Imaging

Shree Nayar
Columbia University

In this talk, we will first present the concept of a computational camera. It is a device that embodies the convergence of the camera and the computer. It uses new optics to select rays from the scene in unusual ways and an appropriate algorithm to process the selected rays. This ability to manipulate light before it is measured and process the measurements before they are presented is a powerful one. We will show computational cameras that can capture depth images, omnidirectional images, and high dynamic range images.

Next, we present a programmable imaging system which is a computational camera whose optics and algorithms can be controlled via software. In particular, we show how spatial light modulators can be used to select and modulate rays from the light field based on the needs of the application at hand. Programmable imaging allows a human user or a vision system to select from, and instantly switch between, a wide range of imaging functionalities. We demonstrate the use of programmable imaging for feature detection, tracking, and object recognition.


Shree K. Nayar received his PhD degree in Electrical and Computer Engineering from the Robotics Institute at Carnegie Mellon University in 1990. He is currently the T. C. Chang Professor of Computer Science at Columbia University. He co-directs the Columbia Vision and Graphics Center. He also heads the Columbia Computer Vision Laboratory (CAVE), which is dedicated to the development of advanced computer vision systems. His research is focused on three areas; the creation of novel vision sensors, the design of physics based models for vision, and the development of algorithms for scene understanding. His work is motivated by applications in the fields of digital imaging, computer graphics, and robotics.

He has received best paper awards at ICCV 1990, ICPR 1994, CVPR 1994, ICCV 1995, CVPR 2000 and CVPR 2004. He is the recipient of the David Marr Prize (1990 and 1995), the David and Lucile Packard Fellowship (1992), the National Young Investigator Award (1993), the NTT Distinguished Scientific Achievement Award (1994), and the Keck Foundation Award for Excellence in Teaching (1995). He has published over 100 scientific papers and holds several patents on inventions related to vision and robotics.