LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability
Electroencephalography (EEG)-based brain-computer interfaces (BCIs) pose a challenge for decoding due to their low spatial resolution and signal-to-noise ratio. Typically, EEG-based recognition of activities and states involves the use of prior neuroscience knowledge to generate quantitative EEG fea...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-08-01
|
| Series: | NeuroImage |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1053811923003609 |
| _version_ | 1827920714355703808 |
|---|---|
| author | Zhengqing Miao Meirong Zhao Xin Zhang Dong Ming |
| author_facet | Zhengqing Miao Meirong Zhao Xin Zhang Dong Ming |
| author_sort | Zhengqing Miao |
| collection | DOAJ |
| description | Electroencephalography (EEG)-based brain-computer interfaces (BCIs) pose a challenge for decoding due to their low spatial resolution and signal-to-noise ratio. Typically, EEG-based recognition of activities and states involves the use of prior neuroscience knowledge to generate quantitative EEG features, which may limit BCI performance. Although neural network-based methods can effectively extract features, they often encounter issues such as poor generalization across datasets, high predicting volatility, and low model interpretability. To address these limitations, we propose a novel lightweight multi-dimensional attention network, called LMDA-Net. By incorporating two novel attention modules designed specifically for EEG signals, the channel attention module and the depth attention module, LMDA-Net is able to effectively integrate features from multiple dimensions, resulting in improved classification performance across various BCI tasks. LMDA-Net was evaluated on four high-impact public datasets, including motor imagery (MI) and P300-Speller, and was compared with other representative models. The experimental results demonstrate that LMDA-Net outperforms other representative methods in terms of classification accuracy and predicting volatility, achieving the highest accuracy in all datasets within 300 training epochs. Ablation experiments further confirm the effectiveness of the channel attention module and the depth attention module. To facilitate an in-depth understanding of the features extracted by LMDA-Net, we propose class-specific neural network feature interpretability algorithms that are suitable for evoked responses and endogenous activities. By mapping the output of the specific layer of LMDA-Net to the time or spatial domain through class activation maps, the resulting feature visualizations can provide interpretable analysis and establish connections with EEG time-spatial analysis in neuroscience. In summary, LMDA-Net shows great potential as a general decoding model for various EEG tasks. |
| first_indexed | 2024-03-13T04:14:50Z |
| format | Article |
| id | doaj.art-a1ea96969f7543208e16e7dbfc4ae022 |
| institution | Directory Open Access Journal |
| issn | 1095-9572 |
| language | English |
| last_indexed | 2024-03-13T04:14:50Z |
| publishDate | 2023-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | NeuroImage |
| spelling | doaj.art-a1ea96969f7543208e16e7dbfc4ae0222023-06-21T06:51:12ZengElsevierNeuroImage1095-95722023-08-01276120209LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretabilityZhengqing Miao0Meirong Zhao1Xin Zhang2Dong Ming3State Key Laboratory of Precision Measuring Technology and Instruments, School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, ChinaCorresponding author.; State Key Laboratory of Precision Measuring Technology and Instruments, School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, ChinaLaboratory of Neural Engineering and Rehabilitation, Department of Biomedical Engineering, School of Precision Instruments and Optoelectronics Engineering, Tianjin University, China; Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, ChinaLaboratory of Neural Engineering and Rehabilitation, Department of Biomedical Engineering, School of Precision Instruments and Optoelectronics Engineering, Tianjin University, China; Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, ChinaElectroencephalography (EEG)-based brain-computer interfaces (BCIs) pose a challenge for decoding due to their low spatial resolution and signal-to-noise ratio. Typically, EEG-based recognition of activities and states involves the use of prior neuroscience knowledge to generate quantitative EEG features, which may limit BCI performance. Although neural network-based methods can effectively extract features, they often encounter issues such as poor generalization across datasets, high predicting volatility, and low model interpretability. To address these limitations, we propose a novel lightweight multi-dimensional attention network, called LMDA-Net. By incorporating two novel attention modules designed specifically for EEG signals, the channel attention module and the depth attention module, LMDA-Net is able to effectively integrate features from multiple dimensions, resulting in improved classification performance across various BCI tasks. LMDA-Net was evaluated on four high-impact public datasets, including motor imagery (MI) and P300-Speller, and was compared with other representative models. The experimental results demonstrate that LMDA-Net outperforms other representative methods in terms of classification accuracy and predicting volatility, achieving the highest accuracy in all datasets within 300 training epochs. Ablation experiments further confirm the effectiveness of the channel attention module and the depth attention module. To facilitate an in-depth understanding of the features extracted by LMDA-Net, we propose class-specific neural network feature interpretability algorithms that are suitable for evoked responses and endogenous activities. By mapping the output of the specific layer of LMDA-Net to the time or spatial domain through class activation maps, the resulting feature visualizations can provide interpretable analysis and establish connections with EEG time-spatial analysis in neuroscience. In summary, LMDA-Net shows great potential as a general decoding model for various EEG tasks.http://www.sciencedirect.com/science/article/pii/S1053811923003609AttentionBrain-computer interface (BCI)Electroencephalography (EEG)Model interpretabilityNeural networks |
| spellingShingle | Zhengqing Miao Meirong Zhao Xin Zhang Dong Ming LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability NeuroImage Attention Brain-computer interface (BCI) Electroencephalography (EEG) Model interpretability Neural networks |
| title | LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability |
| title_full | LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability |
| title_fullStr | LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability |
| title_full_unstemmed | LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability |
| title_short | LMDA-Net:A lightweight multi-dimensional attention network for general EEG-based brain-computer interfaces and interpretability |
| title_sort | lmda net a lightweight multi dimensional attention network for general eeg based brain computer interfaces and interpretability |
| topic | Attention Brain-computer interface (BCI) Electroencephalography (EEG) Model interpretability Neural networks |
| url | http://www.sciencedirect.com/science/article/pii/S1053811923003609 |
| work_keys_str_mv | AT zhengqingmiao lmdanetalightweightmultidimensionalattentionnetworkforgeneraleegbasedbraincomputerinterfacesandinterpretability AT meirongzhao lmdanetalightweightmultidimensionalattentionnetworkforgeneraleegbasedbraincomputerinterfacesandinterpretability AT xinzhang lmdanetalightweightmultidimensionalattentionnetworkforgeneraleegbasedbraincomputerinterfacesandinterpretability AT dongming lmdanetalightweightmultidimensionalattentionnetworkforgeneraleegbasedbraincomputerinterfacesandinterpretability |