Aiming for an FDA-approved System to Enable Speech from Brain Signals
Edward Chang, MD, chair of neurological surgery at UCSF and long-time contributor to BCI technology, envisions this latest advancement, published on August 23, 2023 in Nature, paving the way for an FDA-approved system that translates brain signals into speech. Chang aims to restore a fully developed means of communication, a vital component in patients’ interactions with others. This breakthrough significantly enhances the prospect of providing a genuine solution for patients.
Decoding Brain Signals for Speech and Facial Expressions
In a preceding experiment, Chang’s team effectively decoded the brain signals of a man who experienced a brainstem stroke into text. The current study’s more ambitious undertaking demonstrates the richness of speech by decoding brain signals and capturing movements that stimulate a person’s face during conversation.
Chang implanted a paper-thin, electrode-covered rectangle comprising 253 units over areas of the woman’s brain crucial for speech. These electrodes intercepted brain signals that would control muscles in her tongue, jaw, larynx, and face if not hampered by her stroke. Following weeks of practice, the participant trained the system’s artificial intelligence (AI) algorithms to recognize her unique brain signals for speech.
Key Points:
- Researchers from UC San Francisco and UC Berkeley have developed a brain-computer interface (BCI) that enables a paralyzed woman to communicate via a digital avatar, with speech and facial expressions synthesized directly from brain signals.
- The breakthrough technology can decode brain signals into text at a rate of roughly 80 words per minute, surpassing the performance capabilities of previous technologies.
- The team, led by Edward Chang, MD, utilized a small electrode-covered device placed on the woman’s brain to intercept signals for speech and facial movements, which were then used to train the system’s algorithms.
- For more accurate and faster interpretation, words were decoded from phonemes, which drastically reduced the number of learning categories for the machine, down to just 39 phonemes for the English language.
- The next goal for researchers is to develop a wireless version of the BCI, to provide users with more autonomy and to improve the quality of their social interactions.
Streamlining AI with Phonemes
The team improved the system’s accuracy and speed by devising an approach that decoded words from phonemes, the sub-units of speech forming spoken words. By employing this method, the computer only needed to learn 39 phonemes to interpret any word in English.
Personalized Voice and Avatar Animation
To create a realistic speech experience, the researchers crafted a synthesis algorithm that personalized the voice to match her pre-injury speech, using recordings from her wedding. In collaboration with Speech Graphics, an AI-driven facial animation company, the team synchronized the avatar’s facial movements with signals sent from the woman’s brain as she spoke.
Looking Forward: A Wireless Version
The researchers’ next objective is to create a wireless version of the BCI, allowing users to control their devices and improve their social interactions independently.
