IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG
Objective: The key principle of motor imagery (MI) decoding for electroencephalogram (EEG)-based Brain-Computer Interface (BCI) is to extract task-discriminative features from spectral, spatial, and temporal domains jointly and efficiently, whereas limited, noisy, and non-stationary EEG samples chal...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10070810/ |
| _version_ | 1797805183636340736 |
|---|---|
| author | Jiaheng Wang Lin Yao Yueming Wang |
| author_facet | Jiaheng Wang Lin Yao Yueming Wang |
| author_sort | Jiaheng Wang |
| collection | DOAJ |
| description | Objective: The key principle of motor imagery (MI) decoding for electroencephalogram (EEG)-based Brain-Computer Interface (BCI) is to extract task-discriminative features from spectral, spatial, and temporal domains jointly and efficiently, whereas limited, noisy, and non-stationary EEG samples challenge the advanced design of decoding algorithms. Methods: Inspired by the concept of cross-frequency coupling and its correlation with different behavioral tasks, this paper proposes a lightweight Interactive Frequency Convolutional Neural Network (IFNet) to explore cross-frequency interactions for enhancing representation of MI characteristics. IFNet first extracts spectro-spatial features in low and high-frequency bands, respectively. Then the interplay between the two bands is learned using an element-wise addition operation followed by temporal average pooling. Combined with repeated trial augmentation as a regularizer, IFNet yields spectro-spatio-temporally robust features for the final MI classification. We conduct extensive experiments on two benchmark datasets: the BCI competition IV 2a (BCIC-IV-2a) dataset and the OpenBMI dataset. Results: Compared with state-of-the-art MI decoding algorithms, IFNet achieves significantly superior classification performance on both datasets while improving the winner’s result in BCIC-IV-2a by 11%. Moreover, by conducting sensitivity analysis on decision windows, we show IFNet attains the best trade-off between decoding speed and accuracy. Detailed analysis and visualization verify IFNet can capture the coupling across frequency bands along with the known MI signatures. Conclusion: We demonstrate the effectiveness and superiority of the proposed IFNet for MI decoding. Significance: This study suggests IFNet holds promise for rapid response and accurate control in MI-BCI applications. |
| first_indexed | 2024-03-13T05:48:13Z |
| format | Article |
| id | doaj.art-ca1e36d81ad142c99b180336aa5714f6 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-13T05:48:13Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-ca1e36d81ad142c99b180336aa5714f62023-06-13T20:07:04ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01311900191110.1109/TNSRE.2023.325731910070810IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEGJiaheng Wang0https://orcid.org/0009-0005-0815-0518Lin Yao1https://orcid.org/0000-0003-2065-7280Yueming Wang2https://orcid.org/0000-0001-7742-0722MOE Frontiers Science Center for Brain and Brain–Machine Integration and the College of Computer Science, Zhejiang University, Hangzhou, ChinaMOE Frontiers Science Center for Brain and Brain–Machine Integration and the College of Computer Science, Zhejiang University, Hangzhou, ChinaQiushi Academy for Advanced Studies (QAAS), College of Computer Science, Zhejiang University, Hangzhou, ChinaObjective: The key principle of motor imagery (MI) decoding for electroencephalogram (EEG)-based Brain-Computer Interface (BCI) is to extract task-discriminative features from spectral, spatial, and temporal domains jointly and efficiently, whereas limited, noisy, and non-stationary EEG samples challenge the advanced design of decoding algorithms. Methods: Inspired by the concept of cross-frequency coupling and its correlation with different behavioral tasks, this paper proposes a lightweight Interactive Frequency Convolutional Neural Network (IFNet) to explore cross-frequency interactions for enhancing representation of MI characteristics. IFNet first extracts spectro-spatial features in low and high-frequency bands, respectively. Then the interplay between the two bands is learned using an element-wise addition operation followed by temporal average pooling. Combined with repeated trial augmentation as a regularizer, IFNet yields spectro-spatio-temporally robust features for the final MI classification. We conduct extensive experiments on two benchmark datasets: the BCI competition IV 2a (BCIC-IV-2a) dataset and the OpenBMI dataset. Results: Compared with state-of-the-art MI decoding algorithms, IFNet achieves significantly superior classification performance on both datasets while improving the winner’s result in BCIC-IV-2a by 11%. Moreover, by conducting sensitivity analysis on decision windows, we show IFNet attains the best trade-off between decoding speed and accuracy. Detailed analysis and visualization verify IFNet can capture the coupling across frequency bands along with the known MI signatures. Conclusion: We demonstrate the effectiveness and superiority of the proposed IFNet for MI decoding. Significance: This study suggests IFNet holds promise for rapid response and accurate control in MI-BCI applications.https://ieeexplore.ieee.org/document/10070810/Brain–computer interfacemotor imagerycross-frequency interactionsconvolutional neural networksdata augmentation |
| spellingShingle | Jiaheng Wang Lin Yao Yueming Wang IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG IEEE Transactions on Neural Systems and Rehabilitation Engineering Brain–computer interface motor imagery cross-frequency interactions convolutional neural networks data augmentation |
| title | IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG |
| title_full | IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG |
| title_fullStr | IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG |
| title_full_unstemmed | IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG |
| title_short | IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG |
| title_sort | ifnet an interactive frequency convolutional neural network for enhancing motor imagery decoding from eeg |
| topic | Brain–computer interface motor imagery cross-frequency interactions convolutional neural networks data augmentation |
| url | https://ieeexplore.ieee.org/document/10070810/ |
| work_keys_str_mv | AT jiahengwang ifnetaninteractivefrequencyconvolutionalneuralnetworkforenhancingmotorimagerydecodingfromeeg AT linyao ifnetaninteractivefrequencyconvolutionalneuralnetworkforenhancingmotorimagerydecodingfromeeg AT yuemingwang ifnetaninteractivefrequencyconvolutionalneuralnetworkforenhancingmotorimagerydecodingfromeeg |