A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network
The key to electroencephalography (EEG)-based brain-computer interface (BCI) lies in neural decoding, and its accuracy can be improved by using hybrid BCI paradigms, that is, fusing multiple paradigms. However, hybrid BCIs usually require separate processing processes for EEG signals in each paradig...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2023-12-01
|
| Series: | Brain-Apparatus Communication |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/27706710.2023.2258938 |
| _version_ | 1797641014648766464 |
|---|---|
| author | Wenwei Luo Wanguang Yin Quanying Liu Youzhi Qu |
| author_facet | Wenwei Luo Wanguang Yin Quanying Liu Youzhi Qu |
| author_sort | Wenwei Luo |
| collection | DOAJ |
| description | The key to electroencephalography (EEG)-based brain-computer interface (BCI) lies in neural decoding, and its accuracy can be improved by using hybrid BCI paradigms, that is, fusing multiple paradigms. However, hybrid BCIs usually require separate processing processes for EEG signals in each paradigm, which greatly reduces the efficiency of EEG feature extraction and the generalizability of the model. Here, we propose a two-stream convolutional neural network (TSCNN) based hybrid brain-computer interface. It combines steady-state visual evoked potential (SSVEP) and motor imagery (MI) paradigms. TSCNN automatically learns to extract EEG features in the two paradigms in the training process and improves the decoding accuracy by 25.4% compared with the MI mode, and 2.6% compared with SSVEP mode in the test data. Moreover, the versatility of TSCNN is verified as it provides considerable performance in both single-mode (70.2% for MI, 93.0% for SSVEP) and hybrid-mode scenarios (95.6% for MI-SSVEP hybrid). Our work will facilitate the real-world applications of EEG-based BCI systems. |
| first_indexed | 2024-03-11T13:39:30Z |
| format | Article |
| id | doaj.art-8e3a45ede7f94671b81fc0f4267974bd |
| institution | Directory Open Access Journal |
| issn | 2770-6710 |
| language | English |
| last_indexed | 2024-03-11T13:39:30Z |
| publishDate | 2023-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Brain-Apparatus Communication |
| spelling | doaj.art-8e3a45ede7f94671b81fc0f4267974bd2023-11-02T13:48:32ZengTaylor & Francis GroupBrain-Apparatus Communication2770-67102023-12-012110.1080/27706710.2023.22589382258938A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural networkWenwei Luo0Wanguang Yin1Quanying Liu2Youzhi Qu3Department of Biomedical Engineering, Southern University of Science and TechnologyDepartment of Biomedical Engineering, Southern University of Science and TechnologyDepartment of Biomedical Engineering, Southern University of Science and TechnologyDepartment of Biomedical Engineering, Southern University of Science and TechnologyThe key to electroencephalography (EEG)-based brain-computer interface (BCI) lies in neural decoding, and its accuracy can be improved by using hybrid BCI paradigms, that is, fusing multiple paradigms. However, hybrid BCIs usually require separate processing processes for EEG signals in each paradigm, which greatly reduces the efficiency of EEG feature extraction and the generalizability of the model. Here, we propose a two-stream convolutional neural network (TSCNN) based hybrid brain-computer interface. It combines steady-state visual evoked potential (SSVEP) and motor imagery (MI) paradigms. TSCNN automatically learns to extract EEG features in the two paradigms in the training process and improves the decoding accuracy by 25.4% compared with the MI mode, and 2.6% compared with SSVEP mode in the test data. Moreover, the versatility of TSCNN is verified as it provides considerable performance in both single-mode (70.2% for MI, 93.0% for SSVEP) and hybrid-mode scenarios (95.6% for MI-SSVEP hybrid). Our work will facilitate the real-world applications of EEG-based BCI systems.http://dx.doi.org/10.1080/27706710.2023.2258938brain-computer interface (bci)convolutional neural networks (cnns)electroencephalography (eeg)steady-state visual evoked potential (ssvep)motor imagery (mi) |
| spellingShingle | Wenwei Luo Wanguang Yin Quanying Liu Youzhi Qu A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network Brain-Apparatus Communication brain-computer interface (bci) convolutional neural networks (cnns) electroencephalography (eeg) steady-state visual evoked potential (ssvep) motor imagery (mi) |
| title | A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network |
| title_full | A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network |
| title_fullStr | A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network |
| title_full_unstemmed | A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network |
| title_short | A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network |
| title_sort | hybrid brain computer interface using motor imagery and ssvep based on convolutional neural network |
| topic | brain-computer interface (bci) convolutional neural networks (cnns) electroencephalography (eeg) steady-state visual evoked potential (ssvep) motor imagery (mi) |
| url | http://dx.doi.org/10.1080/27706710.2023.2258938 |
| work_keys_str_mv | AT wenweiluo ahybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT wanguangyin ahybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT quanyingliu ahybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT youzhiqu ahybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT wenweiluo hybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT wanguangyin hybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT quanyingliu hybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork AT youzhiqu hybridbraincomputerinterfaceusingmotorimageryandssvepbasedonconvolutionalneuralnetwork |