Paralyzed Woman Speak Through Digital Avatar: Power of AI

Summary: Researchers at UC San Francisco and UC Berkeley have achieved a groundbreaking breakthrough by developing a brain-computer interface (BCI) allowing a woman suffering from severe paralysis following a brainstem stroke to express herself through a digital avatar. This marks the first instance of speech and facial expressions being synthesized from signals within the brain. Moreover, the system can decode these signals into text at nearly 80 words per minute, significantly improving over existing commercial technology.


Aiming for an FDA-approved System to Enable Speech from Brain Signals

Edward Chang, MD, chair of neurological surgery at UCSF and long-time contributor to BCI technology, envisions this latest advancement, published on August 23, 2023 in Nature, paving the way for an FDA-approved system that translates brain signals into speech. Chang aims to restore a fully developed means of communication, a vital component in patients’ interactions with others. This breakthrough significantly enhances the prospect of providing a genuine solution for patients.

Decoding Brain Signals for Speech and Facial Expressions

Woman in wheelchair interacting with a screen

In a preceding experiment, Chang’s team effectively decoded the brain signals of a man who experienced a brainstem stroke into text. The current study’s more ambitious undertaking demonstrates the richness of speech by decoding brain signals and capturing movements that stimulate a person’s face during conversation.

Chang implanted a paper-thin, electrode-covered rectangle comprising 253 units over areas of the woman’s brain crucial for speech. These electrodes intercepted brain signals that would control muscles in her tongue, jaw, larynx, and face if not hampered by her stroke. Following weeks of practice, the participant trained the system’s artificial intelligence (AI) algorithms to recognize her unique brain signals for speech.

Key Points:

  1. Researchers from UC San Francisco and UC Berkeley have developed a brain-computer interface (BCI) that enables a paralyzed woman to communicate via a digital avatar, with speech and facial expressions synthesized directly from brain signals.
  2. The breakthrough technology can decode brain signals into text at a rate of roughly 80 words per minute, surpassing the performance capabilities of previous technologies.
  3. The team, led by Edward Chang, MD, utilized a small electrode-covered device placed on the woman’s brain to intercept signals for speech and facial movements, which were then used to train the system’s algorithms.
  4. For more accurate and faster interpretation, words were decoded from phonemes, which drastically reduced the number of learning categories for the machine, down to just 39 phonemes for the English language.
  5. The next goal for researchers is to develop a wireless version of the BCI, to provide users with more autonomy and to improve the quality of their social interactions.

Streamlining AI with Phonemes

The team improved the system’s accuracy and speed by devising an approach that decoded words from phonemes, the sub-units of speech forming spoken words. By employing this method, the computer only needed to learn 39 phonemes to interpret any word in English.

Personalized Voice and Avatar Animation

To create a realistic speech experience, the researchers crafted a synthesis algorithm that personalized the voice to match her pre-injury speech, using recordings from her wedding. In collaboration with Speech Graphics, an AI-driven facial animation company, the team synchronized the avatar’s facial movements with signals sent from the woman’s brain as she spoke.

Looking Forward: A Wireless Version

The researchers’ next objective is to create a wireless version of the BCI, allowing users to control their devices and improve their social interactions independently.

Our Vision

AI is often seen as a threat to human presence, but it’s crucial to remember that it’s merely a tool. Like electricity, its potential depends on how we utilize it. At Upbeator, we aim to empower you with knowledge about AI tools, enabling you to harness their wonders instead of fearing them.