A wireless brain-machine interface for real-time speech synthesis.
Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfo...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2009-12-01
|
| Series: | PLoS ONE |
| Online Access: | http://europepmc.org/articles/PMC2784218?pdf=render |
| _version_ | 1819174536320385024 |
|---|---|
| author | Frank H Guenther Jonathan S Brumberg E Joseph Wright Alfonso Nieto-Castanon Jason A Tourville Mikhail Panko Robert Law Steven A Siebert Jess L Bartels Dinal S Andreasen Princewill Ehirim Hui Mao Philip R Kennedy |
| author_facet | Frank H Guenther Jonathan S Brumberg E Joseph Wright Alfonso Nieto-Castanon Jason A Tourville Mikhail Panko Robert Law Steven A Siebert Jess L Bartels Dinal S Andreasen Princewill Ehirim Hui Mao Philip R Kennedy |
| author_sort | Frank H Guenther |
| collection | DOAJ |
| description | Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech.Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer's vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task.Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas. |
| first_indexed | 2024-12-22T20:40:32Z |
| format | Article |
| id | doaj.art-0699a07aadd848a9820d5354e2628d6e |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-22T20:40:32Z |
| publishDate | 2009-12-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-0699a07aadd848a9820d5354e2628d6e2022-12-21T18:13:20ZengPublic Library of Science (PLoS)PLoS ONE1932-62032009-12-01412e821810.1371/journal.pone.0008218A wireless brain-machine interface for real-time speech synthesis.Frank H GuentherJonathan S BrumbergE Joseph WrightAlfonso Nieto-CastanonJason A TourvilleMikhail PankoRobert LawSteven A SiebertJess L BartelsDinal S AndreasenPrincewill EhirimHui MaoPhilip R KennedyBrain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech.Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer's vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task.Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas.http://europepmc.org/articles/PMC2784218?pdf=render |
| spellingShingle | Frank H Guenther Jonathan S Brumberg E Joseph Wright Alfonso Nieto-Castanon Jason A Tourville Mikhail Panko Robert Law Steven A Siebert Jess L Bartels Dinal S Andreasen Princewill Ehirim Hui Mao Philip R Kennedy A wireless brain-machine interface for real-time speech synthesis. PLoS ONE |
| title | A wireless brain-machine interface for real-time speech synthesis. |
| title_full | A wireless brain-machine interface for real-time speech synthesis. |
| title_fullStr | A wireless brain-machine interface for real-time speech synthesis. |
| title_full_unstemmed | A wireless brain-machine interface for real-time speech synthesis. |
| title_short | A wireless brain-machine interface for real-time speech synthesis. |
| title_sort | wireless brain machine interface for real time speech synthesis |
| url | http://europepmc.org/articles/PMC2784218?pdf=render |
| work_keys_str_mv | AT frankhguenther awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT jonathansbrumberg awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT ejosephwright awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT alfonsonietocastanon awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT jasonatourville awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT mikhailpanko awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT robertlaw awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT stevenasiebert awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT jesslbartels awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT dinalsandreasen awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT princewillehirim awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT huimao awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT philiprkennedy awirelessbrainmachineinterfaceforrealtimespeechsynthesis AT frankhguenther wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT jonathansbrumberg wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT ejosephwright wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT alfonsonietocastanon wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT jasonatourville wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT mikhailpanko wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT robertlaw wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT stevenasiebert wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT jesslbartels wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT dinalsandreasen wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT princewillehirim wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT huimao wirelessbrainmachineinterfaceforrealtimespeechsynthesis AT philiprkennedy wirelessbrainmachineinterfaceforrealtimespeechsynthesis |