Our first light field micrographs

September-December, 2005

Specimens: Hernan Espinoza, the antique slide collection of Benjamin Monroe Levoy Microscope design: Marc Levoy
Photography, post-processing and web page: Marc Levoy

All of these micrographs were captured using our first prototype light field microscope (LFM), shown at the top of the project's home page. For more details, see our SIGGRAPH 2006 paper.

Low NA, high depth-of-field

(or AVI)

Light field of insect legs, captured through a microlens array by a Canon 5D digital still camera. The resolution of the captured light field is 200 x 200 microlenses x 15 x 15 pixels per microlens. (The image above is at reduced resolution.) Hence the synthetically computed images at right are 200 x 200 pixels. Click here for the full-res light field.

Synthetic panning sequence. The objective was a Zeiss 25x/0.45NA Plan (dry), which provides 35 degrees of angular parallax on the specimen (about 26 degrees of which is shown here) after accounting for air-specimen refraction. In other words, the pan ranges from 13 degrees left of head-on to 13 degrees right of head-on.

Synthetic focal stack, consisting of 30 slices spaced 8 microns apart in Z, for a total Z-range of 240 microns. However, features are well focused through only 180 microns of this range (in theory, closer to 90 in practice). The depth of field of each slice is about 20 microns.

New:Click at left for an all-focus image computed from the focal stack above, using the algorithm of Agarwala et al., Interactive Digital Photomontage, Proc. SIGGRAPH 2004. It has greater depth of field than images in the focal sequence and is less noisy than images in the panning sequence.

(or AVI)

Light field of embryo mouse lung, captured as described above. (Specimen courtesy of Hernan Espinoza, in Mark Krasnow's laboratory at Stanford University.) The ray diagrams at right are drawn to scale, but without showing air-specimen refraction. The objective is at top (not its real shape) and the original focal plane is at bottom. Click here for the full-res light field.

Synthetic panning sequence. The objective in this example was a Zeiss 16x/0.4NA Neofluor (dry), providing 31 degrees of parallax (26 degrees shown here). Click here for a pan captured using a 40x/0.8NA Achroplan (water) objective. The lateral resolution in the latter pan, set by the microlens spacing, is 4 microns.

Synthetic focal stack, consisting of 78 frames spaced 11.5 microns apart in Z, for a total Z-range of 900 microns. However, features are well focused through only about 200 microns of this range (in practice). The depth of field of each slice is about 34 microns.

New:Click at left for an all-focus image computed from the focal stack above. See the previous insect legs example for more details.

(or AVI)

An older light field of an insect leg, captured by a Canon 20D. The resolution of this light field is 168 x 168 microlenses x 15 x 15 pixels per microlens. Click here for the full-res light field.

Synthetic panning sequence. The objective was a Zeiss 40x/08.NA Achroplan (water), which provides 74 degrees of parallax. However, the ends of the pan are dark due to insufficient angular uniformity of the illumination.

Synthetic focal stack. This is an early result, and the ray diagrams accompanying this and the panning sequence at left are incorrect and should be ignored.

High NA, low depth-of-field

(or AVI)

Light field of a stained silkworm thorax (?). The resolution of the captured light field is 200 x 200 microlenses x 15 x 15 pixels per microlens. (The image above is at reduced resolution.) Click here for full-res.

Image of a blank portion of the slide, taken without moving the microscope, microlens array, or camera. This shows the field limit, objective aperture function, and camera vignetting in the prototype. Click here for full-res.

Grayscale difference of the previous two images. This creates an effect similar to darkfield (peripheral) illumination. Actual darkfield illumination would not produce a usable light field. Click here for full-res.

Synthetic panning sequence. The objective was a Nikon 40x/0.95NA (dry) Plan-Apochromat, which provides 78.6 degrees of angular parallax (71.5 degrees shown here).

Synthetic focal stack, consisting of 45 slices spaced 1.1 micron apart in Z, for a total Z-range of 50 microns. In these panning and focal sequences, the light field has been sharpened slightly in all four dimensions, enhancing its apparent spatial and angular resolutions.

New:Click at left for an all-focus image computed from the focal stack above. See the previous insect legs example for more details.

3D reconstruction (from a single photograph)


Light field of a stained silkworm mandible. The original was in color and was captured under brightfield illumination. The resolution of the captured light field is 200 x 200 x 15 x 15. Click here for full-res.	Synthetic focal stack, consisting of 80 slices spaced 1 micron apart in Z. The slices were subsequently intensity-inverted and cropped to a circle. The objective was a Nikon 40x/1.3NA (oil) Fluor, which provides 120 degrees of angular parallax.	3D reconstruction using nonblind deconvolution and a 3D PSF constructed from the synthetic focusing geometry and empirical measurement of the objective's aperture function. Click here for a perspective flyaround using maximum-intensity projection volume rendering (or AVI)

These reconstructions are performed using the method described in our SIGGRAPH 2006 paper. Here is a screenshot from AutoQuant's deconvolution software package, showing the focal stack at top, the PSF in the middle, and the deconvolution result at bottom. Note the orthogonal cross-sections shown beneath and beside each of the three volumes; these cross-sections are vertically exaggerated 2:1. The slight aliasing visible in the PSF essentially disappears after normalization by the sum of each slice.

The high numerical aperture used here provides more parallax for 3D reconstruction, but the accompanying high magnification limits the resolution of the light field, due to diffraction. (Although the pixel count under each microlens is 15 x 15 pixels, the available resolution is less than half of this.) Limited resolution under each microlens manifests itself as a reduced Z-range that is well focused, as well as a reduced number of resolvable slices within this range. As a result, the best 3D reconstructions will come from low-magnification, high-NA objectives. Such objectives are not hard to build, but they are physically larger than can be accommodated by most current microscopes.

More 3D reconstructions

(or AVI)	Insect legs, from the light field shown above. The input stack was 28 slices spaced 1 micron apart in Z. Colors were inverted before deconvolution to reduce cloudiness in the reconstruction. The objective was a 25x/0.45NA (dry), which provides relatively little parallax. As a result, the top of each leg is poorly resolved. This limitation is inherent to all deconvolution microscopy.
(or AVI)	Grayscale version of the silkworm thorax, from the light field shown above. The input stack was 40 slices spaced 1 micron apart in Z. The objective was a 40x/0.95NA (dry), which provides more parallax than in the example of the insect legs, although not as much as the silkworm thorax.

Compare these 3D reconstructions to the pan sequences shown earlier for these same datasets. The 3D reconstruction is slightly blurrier than the pan, but it shows more 3D structure. It is also less noisy than the pan, since the reconstruction process employs all the available light, whereas each frame in the pan presents only the light passing through a small subwindow of the objective's aperture.