EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization
Due to the limited perceptual field, convolutional neural networks (CNN) only extract local temporal features and may fail to capture long-term dependencies for EEG decoding. In this paper, we propose a compact Convolutional Transformer, named EEG Conformer, to encapsulate local and global features...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9991178/ |
| _version_ | 1797805127656013824 |
|---|---|
| author | Yonghao Song Qingqing Zheng Bingchuan Liu Xiaorong Gao |
| author_facet | Yonghao Song Qingqing Zheng Bingchuan Liu Xiaorong Gao |
| author_sort | Yonghao Song |
| collection | DOAJ |
| description | Due to the limited perceptual field, convolutional neural networks (CNN) only extract local temporal features and may fail to capture long-term dependencies for EEG decoding. In this paper, we propose a compact Convolutional Transformer, named EEG Conformer, to encapsulate local and global features in a unified EEG classification framework. Specifically, the convolution module learns the low-level local features throughout the one-dimensional temporal and spatial convolution layers. The self-attention module is straightforwardly connected to extract the global correlation within the local temporal features. Subsequently, the simple classifier module based on fully-connected layers is followed to predict the categories for EEG signals. To enhance interpretability, we also devise a visualization strategy to project the class activation mapping onto the brain topography. Finally, we have conducted extensive experiments to evaluate our method on three public datasets in EEG-based motor imagery and emotion recognition paradigms. The experimental results show that our method achieves state-of-the-art performance and has great potential to be a new baseline for general EEG decoding. The code has been released in <uri>https://github.com/eeyhsong/EEG-Conformer</uri>. |
| first_indexed | 2024-03-13T05:46:22Z |
| format | Article |
| id | doaj.art-f85d2769b12c404aa18b64e296f93591 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-13T05:46:22Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-f85d2769b12c404aa18b64e296f935912023-06-13T20:09:45ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-013171071910.1109/TNSRE.2022.32302509991178EEG Conformer: Convolutional Transformer for EEG Decoding and VisualizationYonghao Song0https://orcid.org/0000-0003-1700-1133Qingqing Zheng1https://orcid.org/0000-0001-7726-1901Bingchuan Liu2https://orcid.org/0000-0001-5988-6051Xiaorong Gao3https://orcid.org/0000-0003-0499-2740Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, ChinaGuangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaDepartment of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, ChinaDepartment of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, ChinaDue to the limited perceptual field, convolutional neural networks (CNN) only extract local temporal features and may fail to capture long-term dependencies for EEG decoding. In this paper, we propose a compact Convolutional Transformer, named EEG Conformer, to encapsulate local and global features in a unified EEG classification framework. Specifically, the convolution module learns the low-level local features throughout the one-dimensional temporal and spatial convolution layers. The self-attention module is straightforwardly connected to extract the global correlation within the local temporal features. Subsequently, the simple classifier module based on fully-connected layers is followed to predict the categories for EEG signals. To enhance interpretability, we also devise a visualization strategy to project the class activation mapping onto the brain topography. Finally, we have conducted extensive experiments to evaluate our method on three public datasets in EEG-based motor imagery and emotion recognition paradigms. The experimental results show that our method achieves state-of-the-art performance and has great potential to be a new baseline for general EEG decoding. The code has been released in <uri>https://github.com/eeyhsong/EEG-Conformer</uri>.https://ieeexplore.ieee.org/document/9991178/EEG classificationself-attentiontransformerbrain-computer interface (BCI)motor imagery |
| spellingShingle | Yonghao Song Qingqing Zheng Bingchuan Liu Xiaorong Gao EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization IEEE Transactions on Neural Systems and Rehabilitation Engineering EEG classification self-attention transformer brain-computer interface (BCI) motor imagery |
| title | EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization |
| title_full | EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization |
| title_fullStr | EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization |
| title_full_unstemmed | EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization |
| title_short | EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization |
| title_sort | eeg conformer convolutional transformer for eeg decoding and visualization |
| topic | EEG classification self-attention transformer brain-computer interface (BCI) motor imagery |
| url | https://ieeexplore.ieee.org/document/9991178/ |
| work_keys_str_mv | AT yonghaosong eegconformerconvolutionaltransformerforeegdecodingandvisualization AT qingqingzheng eegconformerconvolutionaltransformerforeegdecodingandvisualization AT bingchuanliu eegconformerconvolutionaltransformerforeegdecodingandvisualization AT xiaoronggao eegconformerconvolutionaltransformerforeegdecodingandvisualization |