Researchers from Russian
corporation Neurobotics and the Moscow Institute of Physics and Technology have
found a way to visualize a person's brain activity as actual images mimicking
what they observe in real time. This will enable new post-stroke rehabilitation
devices controlled by brain signals. To develop devices controlled by the brain
and methods for cognitive disorder treatment and post-stroke rehabilitation,
neurobiologists need to understand how the brain encodes information. A key
aspect of this is studying the brain activity of people perceiving visual
information, for example, while watching a video. The existing solutions for
extracting observed images from brain signals either use functional MRI or
analyze the signals picked up via implants directly from neurons. Both methods
have fairly limited applications in clinical practice and everyday life. The
brain-computer interface developed by MIPT and Neurobotics relies on artificial
neural networks and electroencephalography, or EEG, a technique for recording
brain waves via electrodes placed non-invasively on the scalp. By analyzing
brain activity, the system reconstructs the images seen by a person undergoing
EEG in real time.
In the first part of the
experiment, the neurobiologists asked healthy subjects to watch 20 minutes of
10-second YouTube video fragments. The team selected five arbitrary video categories:
abstract shapes, waterfalls, human faces, moving mechanisms and motor sports.
The latter category featured first-person recordings of snowmobile, water
scooter, motorcycle and car races. By analyzing the EEG data, they showed that
the brain wave patterns are distinct for each category of videos. In the second
phase of the experiment, three random categories were selected from the
original five. The researchers developed two neural networks: one for
generating random category-specific images from noise, and another for
generating similar noise from EEG. The team then trained the networks to
operate together in a way that turns the EEG signal into actual images similar
to those the test subjects were observing. To test the system's ability to
visualize brain activity, the subjects were shown previously unseen videos from
the same categories. As they watched, EEGs were recorded and fed to the neural
networks and the system generated convincing images that could be easily
categorized in 90 percent of the cases.
More information: