At Lytro, we often find ourselves explaining what a Light Field is. It’s actually pretty simple to understand if we focus on the fundamentals first, so lets get started.
We live in an illuminated world. The sun, our primary light source, shines brightly on our surroundings. Its directional rays hit the earth’s surface features and structures, reflecting in every direction, bouncing back and forth until they’ve lost their energy.
Whether from the sun or man made light sources, rays of light with their brightness, color and direction, reflect off objects around us, and travel in every direction to create one continuous Light Field. Of all the infinite rays in the Light Field, we can only see those that shine towards our eyes, and pass through our pupils. Our eyes produce neural signals from these rays, informing us of their color, intensity and direction, which in turn, the brain uses to perceive the objects in our world.
As we look at an apple, rays of light are hitting its surface from every direction. They bounce off the skin, the leaves, and the stem, but we only see those rays that reflect towards our eyes and enter through our pupils. We see the apple from two vantage points, our left and right eyes. The side-by-side distance of our eyes gives us a stereoscopic view of the apple, which our brain can use to perceive how near or far the apple is.
A traditional camera perceives the apple the same way one of our eyes works. Light rays pass through the camera’s lens and aperture, and then hit the sensor or film to form the image it captures. The camera’s single lens will only capture the light’s color and intensity, but none of the rays’ angular information.
Stereo cameras have two lenses and sensors, which capture a pair of side-by-side stereoscopic pictures or video of the scene. These are often referred to as 3D images, however they are actually a pair of flat 2D images or video, offset left to right, which trick the eye (and brain) into appearing 3D when viewed through the correct device.
Unlike traditional or stereo cameras, a Light Field camera allows light to be captured from multiple vantage points, left to right, top to bottom, and all points in-between. The Light Field, with the rays’ color, intensity and angular directions are captured to produce images with both color and depth, which can be calculated through the intersection of rays of light in the scene.
Light Field capture systems can be created from an array of cameras, each seeing the scene from a slightly different perspective and contributing a varied vantage point. Every vantage point in the array is then combined to produce Light Field images. To work properly, every camera in the array needs to be perfectly synched, with carefully measured orientation, focal length, shutter timing, and exposure. Setting up a multi-camera array based Light Field capture system requires significant expertise and resources.
At Lytro, we’ve developed a core of Light Field technologies, which enable us to capture a Light Field, compute the ray angles and then replicate that light field in a virtual space.
Inside the Lytro ILLUM, light rays with their color, intensity and angular direction pass through the lens and aperture. Above the sensor, an array of microlenses gathers the rays into tiny image discs. Each disc covers a small portion of the sensor, and the smallest detectable “bundles of rays” correspond to individual pixels on the sensor. The computational engine within the camera processes these bundles of rays, as seen by each pixel on the sensor, and uses a geometric model to calculate the flow of light through a virtual lens. Based on that model and the angular data for each bundle of light, we can generate images of different focal points, variable apertures, shift of perspective and even simulate focal plane effects.
Each lenslet in the Lytro ILLUM’s microlens array provides a different viewpoint of the scene. The sensor underneath that microlens array captures the scene’s Light Field from hundreds of thousands of viewpoints.
The Lytro ILLUM camera provides a convenient Light Field capture platform within a single camera system (one primary lens, one aperture and one shutter) that is easy to use, portable and is perfectly synched by design. In our next article in this series we’ll talk about the Light Field Volume and Light Field video capture.