Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks
Brain-computer interface (BCI) is a direct communication pathway between brain and external devices, and BCI-based prosthetic devices are promising to provide new rehabilitation options for people with motor disabilities. Electrocorticography (ECoG) signals contain rich information correlated with m...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-08-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnins.2018.00555/full |
| _version_ | 1818153413867732992 |
|---|---|
| author | Gang Pan Gang Pan Jia-Jun Li Yu Qi Hang Yu Jun-Ming Zhu Xiao-Xiang Zheng Yue-Ming Wang Shao-Min Zhang |
| author_facet | Gang Pan Gang Pan Jia-Jun Li Yu Qi Hang Yu Jun-Ming Zhu Xiao-Xiang Zheng Yue-Ming Wang Shao-Min Zhang |
| author_sort | Gang Pan |
| collection | DOAJ |
| description | Brain-computer interface (BCI) is a direct communication pathway between brain and external devices, and BCI-based prosthetic devices are promising to provide new rehabilitation options for people with motor disabilities. Electrocorticography (ECoG) signals contain rich information correlated with motor activities, and have great potential in hand gesture decoding. However, most existing decoders use long time windows, thus ignore the temporal dynamics within the period. In this study, we propose to use recurrent neural networks (RNNs) to exploit the temporal information in ECoG signals for robust hand gesture decoding. With RNN's high nonlinearity modeling ability, our method can effectively capture the temporal information in ECoG time series for robust gesture recognition. In the experiments, we decode three hand gestures using ECoG signals of two participants, and achieve an accuracy of 90%. Specially, we investigate the possibility of recognizing the gestures in a time interval as short as possible after motion onsets. Our method rapidly recognizes gestures within 0.5 s after motion onsets with an accuracy of about 80%. Experimental results also indicate that the temporal dynamics is especially informative for effective and rapid decoding of hand gestures. |
| first_indexed | 2024-12-11T14:10:14Z |
| format | Article |
| id | doaj.art-14ab30f1baab4d74ad5b5636501f2e37 |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-11T14:10:14Z |
| publishDate | 2018-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-14ab30f1baab4d74ad5b5636501f2e372022-12-22T01:03:28ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2018-08-011210.3389/fnins.2018.00555353432Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural NetworksGang Pan0Gang Pan1Jia-Jun Li2Yu Qi3Hang Yu4Jun-Ming Zhu5Xiao-Xiang Zheng6Yue-Ming Wang7Shao-Min Zhang8State Key Lab of CAD&CG, Zhejiang University, Hangzhou, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaDepartment of Neurosurgery, The Second Affiliated Hospital of Zhejiang University, Hangzhou, ChinaQiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, ChinaQiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, ChinaQiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, ChinaBrain-computer interface (BCI) is a direct communication pathway between brain and external devices, and BCI-based prosthetic devices are promising to provide new rehabilitation options for people with motor disabilities. Electrocorticography (ECoG) signals contain rich information correlated with motor activities, and have great potential in hand gesture decoding. However, most existing decoders use long time windows, thus ignore the temporal dynamics within the period. In this study, we propose to use recurrent neural networks (RNNs) to exploit the temporal information in ECoG signals for robust hand gesture decoding. With RNN's high nonlinearity modeling ability, our method can effectively capture the temporal information in ECoG time series for robust gesture recognition. In the experiments, we decode three hand gestures using ECoG signals of two participants, and achieve an accuracy of 90%. Specially, we investigate the possibility of recognizing the gestures in a time interval as short as possible after motion onsets. Our method rapidly recognizes gestures within 0.5 s after motion onsets with an accuracy of about 80%. Experimental results also indicate that the temporal dynamics is especially informative for effective and rapid decoding of hand gestures.https://www.frontiersin.org/article/10.3389/fnins.2018.00555/fullbrain-computer interfaceelectrocorticographyneural prosthetic controlneural decodingmotor rehabilitation |
| spellingShingle | Gang Pan Gang Pan Jia-Jun Li Yu Qi Hang Yu Jun-Ming Zhu Xiao-Xiang Zheng Yue-Ming Wang Shao-Min Zhang Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks Frontiers in Neuroscience brain-computer interface electrocorticography neural prosthetic control neural decoding motor rehabilitation |
| title | Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks |
| title_full | Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks |
| title_fullStr | Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks |
| title_full_unstemmed | Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks |
| title_short | Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks |
| title_sort | rapid decoding of hand gestures in electrocorticography using recurrent neural networks |
| topic | brain-computer interface electrocorticography neural prosthetic control neural decoding motor rehabilitation |
| url | https://www.frontiersin.org/article/10.3389/fnins.2018.00555/full |
| work_keys_str_mv | AT gangpan rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT gangpan rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT jiajunli rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT yuqi rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT hangyu rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT junmingzhu rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT xiaoxiangzheng rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT yuemingwang rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks AT shaominzhang rapiddecodingofhandgesturesinelectrocorticographyusingrecurrentneuralnetworks |