Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network
Decoding neural signals of silent reading with Brain-Computer Interface (BCI) techniques presents a fast and intuitive communication method for severely aphasia patients. Electroencephalogram (EEG) acquisition is convenient and easily wearable with high temporal resolution. However, existing EEG-bas...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10379024/ |
| _version_ | 1797354581422768128 |
|---|---|
| author | Chengfang Li Yang Liu Jielin Li Yuhao Miao Jing Liu Liang Song |
| author_facet | Chengfang Li Yang Liu Jielin Li Yuhao Miao Jing Liu Liang Song |
| author_sort | Chengfang Li |
| collection | DOAJ |
| description | Decoding neural signals of silent reading with Brain-Computer Interface (BCI) techniques presents a fast and intuitive communication method for severely aphasia patients. Electroencephalogram (EEG) acquisition is convenient and easily wearable with high temporal resolution. However, existing EEG-based decoding units primarily concentrate on individual words due to their low signal-to-noise ratio, rendering them insufficient for facilitating daily communication. Decoding at the word level is less efficient than decoding at the phrase or sentence level. Furthermore, with the popularity of multilingualism, decoding EEG signals with complex semantics under multiple languages is highly urgent and necessary. To the best of our knowledge, there is currently no research on decoding EEG signals during silent reading of complex semantics, let alone decoding silent reading EEG signals with complex semantics for bilingualism. Moreover, the feasibility of decoding such signals remains to be investigated. In this work, we collect silent reading EEG signals of 9 English Phrases (EP), 7 English Sentences (ES), 10 Chinese Phrases (CP), and 7 Chinese Sentences (CS) from the subject within 26 days. We propose a novel Adaptive Graph Attention Convolution Network (AGACN) for classification. Experimental results demonstrate that our proposed method outperforms state-of-the-art methods, achieving the highest classification accuracy of 54.70%, 62.26%, 44.55%, and 57.14% for silent reading EEG signals of EP, ES, CP, and CS, respectively. Moreover, our results prove the feasibility of complex semantics EEG signal decoding. This work will aid aphasic patients in achieving regular communication while providing novel ideas for neural signal decoding research. |
| first_indexed | 2024-03-08T13:52:29Z |
| format | Article |
| id | doaj.art-ac713a239bce4aa49c00720c14f0161a |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-08T13:52:29Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-ac713a239bce4aa49c00720c14f0161a2024-01-16T00:00:43ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102024-01-013224925810.1109/TNSRE.2023.334898110379024Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional NetworkChengfang Li0https://orcid.org/0000-0001-9077-8032Yang Liu1https://orcid.org/0000-0002-1312-0146Jielin Li2https://orcid.org/0009-0004-7728-2461Yuhao Miao3Jing Liu4https://orcid.org/0000-0002-2819-0200Liang Song5https://orcid.org/0000-0002-8143-9052Academy for Engineering and Technology, Fudan University, Shanghai, ChinaAcademy for Engineering and Technology, Fudan University, Shanghai, ChinaDepartment of Electrical and Computer Engineering, University of Toronto, Toronto, CanadaAcademy for Engineering and Technology, Fudan University, Shanghai, ChinaAcademy for Engineering and Technology, Fudan University, Shanghai, ChinaAcademy for Engineering and Technology, Fudan University, Shanghai, ChinaDecoding neural signals of silent reading with Brain-Computer Interface (BCI) techniques presents a fast and intuitive communication method for severely aphasia patients. Electroencephalogram (EEG) acquisition is convenient and easily wearable with high temporal resolution. However, existing EEG-based decoding units primarily concentrate on individual words due to their low signal-to-noise ratio, rendering them insufficient for facilitating daily communication. Decoding at the word level is less efficient than decoding at the phrase or sentence level. Furthermore, with the popularity of multilingualism, decoding EEG signals with complex semantics under multiple languages is highly urgent and necessary. To the best of our knowledge, there is currently no research on decoding EEG signals during silent reading of complex semantics, let alone decoding silent reading EEG signals with complex semantics for bilingualism. Moreover, the feasibility of decoding such signals remains to be investigated. In this work, we collect silent reading EEG signals of 9 English Phrases (EP), 7 English Sentences (ES), 10 Chinese Phrases (CP), and 7 Chinese Sentences (CS) from the subject within 26 days. We propose a novel Adaptive Graph Attention Convolution Network (AGACN) for classification. Experimental results demonstrate that our proposed method outperforms state-of-the-art methods, achieving the highest classification accuracy of 54.70%, 62.26%, 44.55%, and 57.14% for silent reading EEG signals of EP, ES, CP, and CS, respectively. Moreover, our results prove the feasibility of complex semantics EEG signal decoding. This work will aid aphasic patients in achieving regular communication while providing novel ideas for neural signal decoding research.https://ieeexplore.ieee.org/document/10379024/Brain-computer interfaceEEG signalscomplex semanticssilent readingmultiple languages |
| spellingShingle | Chengfang Li Yang Liu Jielin Li Yuhao Miao Jing Liu Liang Song Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network IEEE Transactions on Neural Systems and Rehabilitation Engineering Brain-computer interface EEG signals complex semantics silent reading multiple languages |
| title | Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network |
| title_full | Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network |
| title_fullStr | Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network |
| title_full_unstemmed | Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network |
| title_short | Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network |
| title_sort | decoding bilingual eeg signals with complex semantics using adaptive graph attention convolutional network |
| topic | Brain-computer interface EEG signals complex semantics silent reading multiple languages |
| url | https://ieeexplore.ieee.org/document/10379024/ |
| work_keys_str_mv | AT chengfangli decodingbilingualeegsignalswithcomplexsemanticsusingadaptivegraphattentionconvolutionalnetwork AT yangliu decodingbilingualeegsignalswithcomplexsemanticsusingadaptivegraphattentionconvolutionalnetwork AT jielinli decodingbilingualeegsignalswithcomplexsemanticsusingadaptivegraphattentionconvolutionalnetwork AT yuhaomiao decodingbilingualeegsignalswithcomplexsemanticsusingadaptivegraphattentionconvolutionalnetwork AT jingliu decodingbilingualeegsignalswithcomplexsemanticsusingadaptivegraphattentionconvolutionalnetwork AT liangsong decodingbilingualeegsignalswithcomplexsemanticsusingadaptivegraphattentionconvolutionalnetwork |