A Wireless Brain-Machine Interface for Real-Time Speech Synthesis
Background: Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing proce...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Public Library of Science
2011
|
| Online Access: | http://hdl.handle.net/1721.1/64994 https://orcid.org/0000-0002-1418-8580 |
| _version_ | 1826191220994473984 |
|---|---|
| author | Guenther, Frank H. Brumberg, Jonathan S. Wright, E. Joseph Nieto-Castanon, Alfonso Tourville, Jason A. Panko, Mikhail Law, Robert Siebert, Steven A. Bartels, Jess L. Andreasen, Dinal S. Ehirim, Princewill Mao, Hui Kennedy, Philip R. |
| author2 | Harvard University--MIT Division of Health Sciences and Technology |
| author_facet | Harvard University--MIT Division of Health Sciences and Technology Guenther, Frank H. Brumberg, Jonathan S. Wright, E. Joseph Nieto-Castanon, Alfonso Tourville, Jason A. Panko, Mikhail Law, Robert Siebert, Steven A. Bartels, Jess L. Andreasen, Dinal S. Ehirim, Princewill Mao, Hui Kennedy, Philip R. |
| author_sort | Guenther, Frank H. |
| collection | MIT |
| description | Background: Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech.
Methodology/Principal Findings: Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer's vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task.
Conclusions/Significance: Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas. |
| first_indexed | 2024-09-23T08:52:30Z |
| format | Article |
| id | mit-1721.1/64994 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:52:30Z |
| publishDate | 2011 |
| publisher | Public Library of Science |
| record_format | dspace |
| spelling | mit-1721.1/649942022-09-26T08:52:47Z A Wireless Brain-Machine Interface for Real-Time Speech Synthesis Guenther, Frank H. Brumberg, Jonathan S. Wright, E. Joseph Nieto-Castanon, Alfonso Tourville, Jason A. Panko, Mikhail Law, Robert Siebert, Steven A. Bartels, Jess L. Andreasen, Dinal S. Ehirim, Princewill Mao, Hui Kennedy, Philip R. Harvard University--MIT Division of Health Sciences and Technology Picower Institute for Learning and Memory Guenther, Frank H. Guenther, Frank H. Background: Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech. Methodology/Principal Findings: Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer's vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task. Conclusions/Significance: Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas. National Institute on Deafness and Other Communication Disorders (U.S.) (Grant R44-DC007050) National Institute on Deafness and Other Communication Disorders (U.S.) (Grant R01-DC007683) National Institute on Deafness and Other Communication Disorders (U.S.) (Grant R01-DC002852) Center of Excellence for Learning in Education, Science, and Technology (SBE-0354378) 2011-07-29T20:22:13Z 2011-07-29T20:22:13Z 2009-12 2009-07 Article http://purl.org/eprint/type/JournalArticle 1932-6203 http://hdl.handle.net/1721.1/64994 Guenther, Frank H. et al. “A Wireless Brain-Machine Interface for Real-Time Speech Synthesis.” Ed. Eshel Ben-Jacob. PLoS ONE 4.12 (2009) : e8218. © 2009 Guenther et al. https://orcid.org/0000-0002-1418-8580 en_US http://dx.doi.org/10.1371/journal.pone.0008218 PLoS ONE Creative Commons Attribution http://creativecommons.org/licenses/by/2.5/ application/pdf Public Library of Science PLoS |
| spellingShingle | Guenther, Frank H. Brumberg, Jonathan S. Wright, E. Joseph Nieto-Castanon, Alfonso Tourville, Jason A. Panko, Mikhail Law, Robert Siebert, Steven A. Bartels, Jess L. Andreasen, Dinal S. Ehirim, Princewill Mao, Hui Kennedy, Philip R. A Wireless Brain-Machine Interface for Real-Time Speech Synthesis |
| title | A Wireless Brain-Machine Interface for Real-Time Speech Synthesis |
| title_full | A Wireless Brain-Machine Interface for Real-Time Speech Synthesis |
| title_fullStr | A Wireless Brain-Machine Interface for Real-Time Speech Synthesis |
| title_full_unstemmed | A Wireless Brain-Machine Interface for Real-Time Speech Synthesis |
| title_short | A Wireless Brain-Machine Interface for Real-Time Speech Synthesis |
| title_sort | wireless brain machine interface for real time speech synthesis |
| url | http://hdl.handle.net/1721.1/64994 https://orcid.org/0000-0002-1418-8580 |
| work_keys_str_mv | AT guentherfrankh awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT brumbergjonathans awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT wrightejoseph awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT nietocastanonalfonso awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT tourvillejasona awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT pankomikhail awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT lawrobert awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT siebertstevena awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT bartelsjessl awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT andreasendinals awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT ehirimprincewill awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT maohui awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT kennedyphilipr awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT guentherfrankh wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT brumbergjonathans wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT wrightejoseph wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT nietocastanonalfonso wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT tourvillejasona wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT pankomikhail wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT lawrobert wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT siebertstevena wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT bartelsjessl wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT andreasendinals wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT ehirimprincewill wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT maohui wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT kennedyphilipr wirelessbrainmachineinterfaceforrealtimespeechsynthesis |