Oculus Patent for Light Field Cameras for Eye Tracking

Facebook’s Oculus patented an eye tracking technique which uses light field cameras inside the headset. Most previous eye tracking systems used a regular or infrared camera combined with an IR illuminator to keep the eyes lit. A light field camera differs from a regular camera in that it also captures the direction that light is travelling. This directional information can be used to understand the depth of the image, and thus 3D shape of the eye, instead of just the color and brightness. By knowing the 3D shape of the eye, the system can find out where the pupil is relative to the eye itself, and thus a more accurate estimation of the user’s gaze direction than with just the apparent 2D shape of the pupil. 

Eye tracking can greatly enhance the feeling of social presence in multiplayer VR, but its most promising use case is foveated rendering. Foveated rendering is when only what you’re looking at is drawn at full resolution while the rest of the scene in your peripheral view is rendered in low detail. This works because human vision is only high detail in the very center. To see this for yourself, look at some text in the room you’re in right now then look just a few feet to the side of it and try to read it again. Foveated rendering should one day enable much higher resolution VR headsets without requiring an expensive top of the line graphics card. Finding a way to make it work reliably is crucial to the future of VR.

More information:

https://uploadvr.com/oculus-patents-light-field-camera-eye-tracking/

iMareCulture 24M Project Meeting

Between the 6th to the 7th of November 2018 the 24 month (annual) meeting of iMareCulture EU project took place at the Istituto Superiore per la Conservazione ed il Restauro in Rome, Italy.

During the meeting important issues regarding the progress of the project where discussed including (but not limited) to: virtual reality serious games, underwater augmented reality and storytelling.

More information:

https://imareculture.weebly.com/

Digital Actors for Making Movies After Death

Visual-effects company Digital Domain, which has worked on major pictures like Avengers: Infinity War and Ready Player One, has also taken on individual celebrities as clients, though it hasn’t publicized the service. The suite of services that the company offers actors includes a range of different scans to capture their famous faces from every conceivable angle, making it simpler to re-create them in the future. Using hundreds of custom LED lights arranged in a sphere, numerous images can be recorded in seconds capturing what the person’s face looks like lit from every angle—and right down to the pores.

The lights can also emit different colors, emulating a variety of outdoor conditions where the digital human may be placed. This allows for more detailed and accurately colored, shaded, and reflective skin. They capture basically how the subdermal blood flow will change in the face. Even with all the advances in CGI, digitally re-created people don’t look perfect yet. Therefore, detailed and sometimes frame-by-frame adjustments have to be made if they aren’t picked up by the model generated from the live actors. But the special effects continue to improve, and with more actors preserving their digital likeness at a young age things could get easier.

More information:

https://www.technologyreview.com/s/612291/actors-are-digitally-preserving-themselves-to-continue-their-careers-beyond-the-grave/

Apple's Secret AR and VR Project

Apple has been exploring virtual reality and augmented reality technologies for more than 10 years based on patent filings, but with virtual and augmented reality exploding in popularity with the launch of ARKit, Apple's dabbling may be growing more serious and could lead to an actual dedicated AR/VR product in the not-too-distant future. Apple is rumored to have a secret research unit comprising hundreds of employees working on AR and VR, exploring ways the emerging technologies could be used in future Apple products. VR/AR hiring has ramped up and Apple has acquired multiple AR/VR companies. Apple is working on an incredibly powerful AR/VR headset that's not quite like anything else on the market. It is said to feature an 8K display for each eye that would be untethered from either a computer or a smartphone, and it would work with both virtual and augmented reality applications.

Rather than relying on a connection to a smartphone or a computer, the headset would connect to a dedicated box using a high-speed short-range wireless technology called 60GHz WiGig. The box would be powered by a custom 5-nanometer Apple processor that's more powerful than anything currently available. At the current time, the box apparently resembles a PC tower, but it won't be an actual Mac computer. To use the headset, users will not need to install special cameras in a room to detect their location as with some available VR headsets. All of the technology will be built into the headset and the box. Apple is said to be aiming to launch this AR/VR headset in 2020, but the company's plans could be delayed or scrapped. Apart of a powerful AR/VR headset, there are rumors pointing towards work on augmented reality smart glasses. As for input methods, Apple is considering touch panels, voice activation, and head gestures.

More information:

https://www.macrumors.com/roundup/apple-vr-project/

Inventions 2018 Paper

A few weeks ago, HCI Lab researchers and colleagues from Terpsichore EU project have published a peer-review paper at the Journal of Frontiers entitled "Digitization and Visualization of Folk Dances in Cultural Heritage: A Review". The paper presents an overview of the techniques used for the recording of dance moves. The different ways of visualization and giving the feedback to the user are reviewed as well as ways of performance evaluation. This paper reviews advances in digitization and visualization of folk dances from 2000 to 2018.

Folk dances represent part of cultural heritage and their preservation for the next generations appears of major importance. Digitization and visualization of folk dances form an increasingly active research area in computer science. In parallel to the rapidly advancing technologies, new ways for learning folk dances are explored, making the digitization and visualization of assorted folk dances for learning purposes using different equipment possible. Along with challenges and limitations, solutions that can assist the learning process are proposed.

More information:

https://www.mdpi.com/2411-5134/3/4/72