Facebook Reality Labs (FRL) believes AR and VR will be the primary way people work, play, and connect in the future. Dubbed ‘Codec Avatars’, the Pittsburgh office is using what they call groundbreaking 3D capture technology and AI systems to generate lifelike virtual avatars that could provide the basis of a quick and easy personal avatar creator of the future. The company says at this point these sorts of real-time, photorealistic avatars require quite the gear to achieve. The lab’s two capture studios—one for the face, and one for the body—are admittedly both large and impractical at this point.
The ultimate goal however is to achieve all of this through lightweight headsets, although FRL Pittsburgh currently uses its own prototype Head Mounted Capture systems (HMCs) equipped with cameras, accelerometers, gyroscopes, magnetometers, infrared lighting, and microphones to capture the full range of human expression. Using a small group of participants, the lab captures 1GB of data per second in effort to create a database of physical traits. In the future, the hope is consumers will be able to create their own avatars without a capture studio and without much data either.