A new method from MIT creates holograms for virtual reality, 3D printing and other potential applications in real-time. The team designed a convolutional neural network, a processing technique that uses a chain of trainable tensors to roughly mimic how humans process visual information. Training a neural network typically requires a large, high-quality dataset, which did not previously exist for 3D holograms. The team built a custom database of 4,000 pairs of computer-generated images. Each pair matched a picture (including color and depth information for each pixel) with its corresponding hologram.
To create the holograms in the new database, the researchers used scenes with complex and variable shapes and colors, with the depth of pixels distributed evenly from the background to the foreground, and with a new set of physics-based calculations to handle occlusion. That approach resulted in photorealistic training data. Next, the algorithm got to work. By learning from each image pair, the tensor network tweaked the parameters of its own calculations, successively enhancing its ability to create holograms.
More information: