11 November 2018

Oculus Patent for Light Field Cameras for Eye Tracking

Facebook’s Oculus patented an eye tracking technique which uses light field cameras inside the headset. Most previous eye tracking systems used a regular or infrared camera combined with an IR illuminator to keep the eyes lit. A light field camera differs from a regular camera in that it also captures the direction that light is travelling. This directional information can be used to understand the depth of the image, and thus 3D shape of the eye, instead of just the color and brightness. By knowing the 3D shape of the eye, the system can find out where the pupil is relative to the eye itself, and thus a more accurate estimation of the user’s gaze direction than with just the apparent 2D shape of the pupil. 


Eye tracking can greatly enhance the feeling of social presence in multiplayer VR, but its most promising use case is foveated rendering. Foveated rendering is when only what you’re looking at is drawn at full resolution while the rest of the scene in your peripheral view is rendered in low detail. This works because human vision is only high detail in the very center. To see this for yourself, look at some text in the room you’re in right now then look just a few feet to the side of it and try to read it again. Foveated rendering should one day enable much higher resolution VR headsets without requiring an expensive top of the line graphics card. Finding a way to make it work reliably is crucial to the future of VR.

More information: