A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding
Motor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10275093/ |
| _version_ | 1797656253512548352 |
|---|---|
| author | Hongyi Zhi Zhuliang Yu Tianyou Yu Zhenghui Gu Jian Yang |
| author_facet | Hongyi Zhi Zhuliang Yu Tianyou Yu Zhenghui Gu Jian Yang |
| author_sort | Hongyi Zhi |
| collection | DOAJ |
| description | Motor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and lead to inaccurate decoding outcomes due to the redundant information. To address this limitation and make full use of the multi-domain EEG features, a multi-domain temporal-spatial-frequency convolutional neural network (TSFCNet) is proposed for MI decoding. The proposed network provides a novel mechanism that utilize the spatial and temporal EEG features combined with frequency and time-frequency characteristics. This network enables powerful feature extraction without complicated network structure. Specifically, the TSFCNet first employs the MixConv-Residual block to extract multiscale temporal features from multi-band filtered EEG data. Next, the temporal-spatial-frequency convolution block implements three shallow, parallel and independent convolutional operations in spatial, frequency and time-frequency domain, and captures high discriminative representations from these domains respectively. Finally, these features are effectively aggregated by average pooling layers and variance layers, and the network is trained with the joint supervision of the cross-entropy and the center loss. Our experimental results show that the TSFCNet outperforms the state-of-the-art models with superior classification accuracy and kappa values (82.72% and 0.7695 for dataset BCI competition IV 2a, 86.39% and 0.7324 for dataset BCI competition IV 2b). These competitive results demonstrate that the proposed network is promising for enhancing the decoding performance of MI BCIs. |
| first_indexed | 2024-03-11T17:26:41Z |
| format | Article |
| id | doaj.art-7104c28fa6764c5f9b94959bea1b31c7 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-11T17:26:41Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-7104c28fa6764c5f9b94959bea1b31c72023-10-18T23:00:08ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01313988399810.1109/TNSRE.2023.332332510275093A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery DecodingHongyi Zhi0https://orcid.org/0000-0001-9601-7787Zhuliang Yu1https://orcid.org/0000-0002-5502-8321Tianyou Yu2https://orcid.org/0000-0002-1805-5339Zhenghui Gu3https://orcid.org/0000-0001-9365-2953Jian Yang4https://orcid.org/0000-0002-2984-8836School of Automation Science and Engineering, South China University of Technology, Guangdong, ChinaSchool of Automation Science and Engineering, South China University of Technology, Guangdong, ChinaSchool of Automation Science and Engineering, South China University of Technology, Guangdong, ChinaSchool of Automation Science and Engineering, South China University of Technology, Guangdong, ChinaSchool of Automation Science and Engineering, South China University of Technology, Guangdong, ChinaMotor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and lead to inaccurate decoding outcomes due to the redundant information. To address this limitation and make full use of the multi-domain EEG features, a multi-domain temporal-spatial-frequency convolutional neural network (TSFCNet) is proposed for MI decoding. The proposed network provides a novel mechanism that utilize the spatial and temporal EEG features combined with frequency and time-frequency characteristics. This network enables powerful feature extraction without complicated network structure. Specifically, the TSFCNet first employs the MixConv-Residual block to extract multiscale temporal features from multi-band filtered EEG data. Next, the temporal-spatial-frequency convolution block implements three shallow, parallel and independent convolutional operations in spatial, frequency and time-frequency domain, and captures high discriminative representations from these domains respectively. Finally, these features are effectively aggregated by average pooling layers and variance layers, and the network is trained with the joint supervision of the cross-entropy and the center loss. Our experimental results show that the TSFCNet outperforms the state-of-the-art models with superior classification accuracy and kappa values (82.72% and 0.7695 for dataset BCI competition IV 2a, 86.39% and 0.7324 for dataset BCI competition IV 2b). These competitive results demonstrate that the proposed network is promising for enhancing the decoding performance of MI BCIs.https://ieeexplore.ieee.org/document/10275093/Brain–computer interface (BCI)electroencephalography (EEG)motor imagery (MI)convolutional neural network (CNN)center loss |
| spellingShingle | Hongyi Zhi Zhuliang Yu Tianyou Yu Zhenghui Gu Jian Yang A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding IEEE Transactions on Neural Systems and Rehabilitation Engineering Brain–computer interface (BCI) electroencephalography (EEG) motor imagery (MI) convolutional neural network (CNN) center loss |
| title | A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding |
| title_full | A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding |
| title_fullStr | A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding |
| title_full_unstemmed | A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding |
| title_short | A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding |
| title_sort | multi domain convolutional neural network for eeg based motor imagery decoding |
| topic | Brain–computer interface (BCI) electroencephalography (EEG) motor imagery (MI) convolutional neural network (CNN) center loss |
| url | https://ieeexplore.ieee.org/document/10275093/ |
| work_keys_str_mv | AT hongyizhi amultidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT zhuliangyu amultidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT tianyouyu amultidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT zhenghuigu amultidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT jianyang amultidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT hongyizhi multidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT zhuliangyu multidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT tianyouyu multidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT zhenghuigu multidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding AT jianyang multidomainconvolutionalneuralnetworkforeegbasedmotorimagerydecoding |