We have developed a camera that enables a user to capture (with the press of a button)
a stack of images of a scene that correspond to different focus settings. This focal
sweep camera (Fosera, for short) uses a high-speed image sensor that is
translated with respect to the lens while images are recorded. The captured stack of images
is referred to as a duration focal stack - images of a possibly dynamic scene captured
while the plane of focus is swept through it. We have derived a condition
that determines, for any given imaging system, the sensor speed and shortest duration
needed to sweep the depth of field of the imaging system through a desired depth range.
In our experiments we have used durations between 200 msec and 500 msec to capture stacks
that have between 24 and 60 images. While we have used sensor actuation in our current
system, focal sweep can be implemented using other means, such as, the use of a liquid
lens whose focal length is varied as a function of time.
While the focal stack produced by a light field camera (such as Lytro) corresponds to a single
instant of time, the focal stack captured by Fosera corresponds to a finite duration
of time and hence includes scene motion. Put a different way, while the Lytro trades off spatial
resolution to capture an instantaneous focal stack, Fosera trades off temporal resolution to maintain
high spatial resolution. We have explored the information embedded within a duration focal stack
and developed an algorithm for computing a depth map which, in the case of a dynamic scene,
includes depth values that correspond to different instants of time. This depth map is used as an
index map to facilitate post-focusing over space and time - we refer to this as space-time refocusing.
To show our refocusing results we have developed a click-to-refocus viewer for duration focal
stacks. While our current viewer uses a single index map for refocusing, we have also developed
a method that tracks objects in the focal stack to compute a different index map for each
image in the focal stack. In this setting, clicking on a moving object takes the user to
the image in which that object comes into focus.
This work was supported in parts by ONR (Grant Nos. N00014-11-1-0285 and N00014-08-1-0929)
and DARPA (Grant No. W911NF-10-1-0214).