Vocalize the Voiceless: AI Makes a Paralyzed Woman Speak

Researchers at UC San Francisco and UC Berkeley have achieved a remarkable breakthrough by developing a brain-computer interface (BCI) that empowers a woman suffering from severe paralysis to communicate through a digital avatar (1^✔ ✔Trusted Source
Novel brain implant helps paralyzed woman speak using a digital avatar

Go to source). This pioneering feat marks the first instance where speech and facial expressions have been synthesized directly from brain signals. The digital avatar’s voice was personalized to mirror the user’s voice pre-injury, and facial animations were driven by software that interpreted the brain’s signals for various facial expressions.

Furthermore, the system can decode these brain signals into text at a remarkable speed of nearly 80 words per minute, presenting a significant advancement compared to existing commercial technologies.

Advertisement

Study Towards Replenishing Lost Speech

Dr. Edward Chang, MD, chair of neurological surgery at UCSF and a long-time contributor to the BCI field, envisions this recent breakthrough as a major stride towards obtaining FDA approval for a brain signal-enabled speech communication system in the near future.

This research development holds the potential to revolutionize communication for individuals with severe paralysis. Chang emphasizes, “Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others.”

He is associated with the UCSF Weill Institute for Neuroscience and holds the distinguished position of the Jeanne Robertson Distinguished Professor in Psychiatry. He believes that these advancements bring them closer to a practical solution for patients seeking effective communication avenues.

Advertisement

Elevating Brain Signal Decoding

This breakthrough builds upon Chang's previous work, which demonstrated the decoding of brain signals into text for an individual who had suffered a brainstem stroke years earlier.

The current study, however, ventures into more ambitious territory: decoding brain signals to reproduce the richness of speech and the following facial expressions that characterize human conversation.

Chang's approach involved implanting a thin rectangular array of 253 electrodes onto the woman's brain's surface, strategically placed over regions crucial for speech production.

These electrodes intercepted brain signals that would have controlled muscles responsible for speech and facial expressions if not for the stroke. These signals were then relayed through a cable connected to a computer bank.

Advertisement

Training the Brain-Computer Interface

Over several weeks, the participant collaborated with the research team to train the artificial intelligence algorithms of the system to recognize her distinct brain signals associated with speech.

This training regimen encompassed repetitive enunciation of phrases from a 1,024-word conversational lexicon until the computer successfully correlated brain activity patterns with specific sounds.

Rather than teaching the AI to identify complete words, the researchers devised a system that deciphers words from phonemes, the elemental speech units analogous to letters in written language.

For instance, the word "Hello" consists of four phonemes: "HH," "AH," "L," and "OW." By adopting this approach, the AI only needed to learn 39 phonemes to decode any English word. This innovation not only boosted the system's accuracy but also tripled its speed.

Pathway to a More Natural and Fluid Communication

Sean Metzger and Alex Silva, graduate students from the joint Bioengineering Program at UC Berkeley and UCSF, led the development of the text decoder. Metzger highlighted the importance of accuracy, speed, and vocabulary in facilitating near-normal conversations. He stressed, "It’s what gives a user the potential, in time, to communicate almost as fast as we do, and to have much more naturalistic and normal conversations."

Creating the synthesized voice entailed formulating an algorithm for speech synthesis, meticulously customized to mirror the participant's pre-injury voice. The team used a recording of her speaking at her wedding to personalize the voice.

The digital avatar's facial expressions were animated using specialized software from Speech Graphics, a company specializing in AI-driven facial animation.

The Road Ahead for Brain-Computer Interfaces

The intersection of neuroscience and technology has led to groundbreaking research in the field of nervous systems, particularly in the realm of brain-computer interfaces (BCIs). BCIs enable the translation of intricate electrical signals, originating from the brain's electrical activity, into a tangible communication channel.

Employing machine learning techniques, researchers delve into the nuances of body language, including micro expressions and eyebrow raises, to decode and interpret these signals. This symbiotic interaction between human brain and machine has the potential to revolutionize communication, particularly for individuals facing challenges like amyotrophic lateral sclerosis (ALS) that impede conventional forms of expression.

From capturing the intricacies of the human face to producing speech via the manipulation of vocal tracts and vocal folds, BCIs hold the promise of translating electrical activity into text-to-speech formats in real-time, thus enabling individuals to convey emotions and messages through a novel speech synthesizer.

While this breakthrough signifies a monumental leap forward in brain-computer interfaces, the long-term implications and potential of such technology are still unfolding. Nevertheless, the convergence of neuroscience and artificial intelligence offers unprecedented possibilities in restoring communication and improving the lives of those with severe disabilities.

Reference:

1. Novel brain implant helps paralyzed woman speak using a digital avatar - (https://data.berkeley.edu/news/novel-brain-implant-helps-paralyzed-woman-speak-using-digital-avatar)

Source-Medindia