A team of researchers at Columbia
University has developed a speech brain-computer interface system that
translates brain signals into intelligible, recognizable speech. By monitoring
someone’s brain activity, the system can reconstruct the words a person hears
with unprecedented clarity. This could lead to new ways for computers to
communicate directly with the brain, and lays the groundwork for helping people
who cannot speak. Early efforts to decode brain signals researchers focused on
simple computer models that analyzed spectrograms, which are visual
representations of sound frequencies. But because this approach has failed to
produce anything resembling intelligible speech, the team turned instead to a
vocoder, a computer algorithm that can synthesize speech after being trained on
recordings of people talking. This is the same technology used by Amazon Echo
and Apple Siri to give verbal responses to our questions.
Researchers asked epilepsy
patients already undergoing brain surgery to listen to sentences spoken by
different people, while they measured patterns of brain activity. These neural
patterns trained the vocoder. Next, they asked those same patients to listen to
speakers reciting digits between 0 to 9, while recording brain signals that
could then be run through the vocoder. The sound produced by the vocoder in
response to those signals was analyzed and cleaned up by neural networks, a
type of artificial intelligence that mimics the structure of neurons in the
biological brain. The end result was a robotic-sounding voice reciting a
sequence of numbers. To test the accuracy of the recording, the scientists
tasked individuals to listen to the recording and report what they heard. They
found that people could understand and repeat the sounds about 75% of the time,
which is well above and beyond any previous attempts.
More information: