Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation
Emotion recognition is crucial in understanding human affective states with various applications. Electroencephalography (EEG)—a non-invasive neuroimaging technique that captures brain activity—has gained attention in emotion recognition. However, existing EEG-based emotion recognition systems are l...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Brain Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3425/13/9/1326 |
| _version_ | 1797581062988103680 |
|---|---|
| author | Chenguang Gao Hirotaka Uchitomi Yoshihiro Miyake |
| author_facet | Chenguang Gao Hirotaka Uchitomi Yoshihiro Miyake |
| author_sort | Chenguang Gao |
| collection | DOAJ |
| description | Emotion recognition is crucial in understanding human affective states with various applications. Electroencephalography (EEG)—a non-invasive neuroimaging technique that captures brain activity—has gained attention in emotion recognition. However, existing EEG-based emotion recognition systems are limited to specific sensory modalities, hindering their applicability. Our study innovates EEG emotion recognition, offering a comprehensive framework for overcoming sensory-focused limits and cross-sensory challenges. We collected cross-sensory emotion EEG data using multimodal emotion simulations (three sensory modalities: audio/visual/audio-visual with two emotion states: pleasure or unpleasure). The proposed framework—filter bank adversarial domain adaptation Riemann method (FBADR)—leverages filter bank techniques and Riemannian tangent space methods for feature extraction from cross-sensory EEG data. Compared with Riemannian methods, filter bank and adversarial domain adaptation could improve average accuracy by 13.68% and 8.36%, respectively. Comparative analysis of classification results proved that the proposed FBADR framework achieved a state-of-the-art cross-sensory emotion recognition performance and reached an average accuracy of 89.01% ± 5.06%. Moreover, the robustness of the proposed methods could ensure high cross-sensory recognition performance under a signal-to-noise ratio (SNR) ≥ 1 dB. Overall, our study contributes to the EEG-based emotion recognition field by providing a comprehensive framework that overcomes limitations of sensory-oriented approaches and successfully tackles the difficulties of cross-sensory situations. |
| first_indexed | 2024-03-10T22:58:49Z |
| format | Article |
| id | doaj.art-eb648058e10a4665b5384bf54525da81 |
| institution | Directory Open Access Journal |
| issn | 2076-3425 |
| language | English |
| last_indexed | 2024-03-10T22:58:49Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Brain Sciences |
| spelling | doaj.art-eb648058e10a4665b5384bf54525da812023-11-19T09:49:21ZengMDPI AGBrain Sciences2076-34252023-09-01139132610.3390/brainsci13091326Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain AdaptationChenguang Gao0Hirotaka Uchitomi1Yoshihiro Miyake2Department of Computer Science, Tokyo Institute of Technology, Yokohama 226-8502, JapanDepartment of Computer Science, Tokyo Institute of Technology, Yokohama 226-8502, JapanDepartment of Computer Science, Tokyo Institute of Technology, Yokohama 226-8502, JapanEmotion recognition is crucial in understanding human affective states with various applications. Electroencephalography (EEG)—a non-invasive neuroimaging technique that captures brain activity—has gained attention in emotion recognition. However, existing EEG-based emotion recognition systems are limited to specific sensory modalities, hindering their applicability. Our study innovates EEG emotion recognition, offering a comprehensive framework for overcoming sensory-focused limits and cross-sensory challenges. We collected cross-sensory emotion EEG data using multimodal emotion simulations (three sensory modalities: audio/visual/audio-visual with two emotion states: pleasure or unpleasure). The proposed framework—filter bank adversarial domain adaptation Riemann method (FBADR)—leverages filter bank techniques and Riemannian tangent space methods for feature extraction from cross-sensory EEG data. Compared with Riemannian methods, filter bank and adversarial domain adaptation could improve average accuracy by 13.68% and 8.36%, respectively. Comparative analysis of classification results proved that the proposed FBADR framework achieved a state-of-the-art cross-sensory emotion recognition performance and reached an average accuracy of 89.01% ± 5.06%. Moreover, the robustness of the proposed methods could ensure high cross-sensory recognition performance under a signal-to-noise ratio (SNR) ≥ 1 dB. Overall, our study contributes to the EEG-based emotion recognition field by providing a comprehensive framework that overcomes limitations of sensory-oriented approaches and successfully tackles the difficulties of cross-sensory situations.https://www.mdpi.com/2076-3425/13/9/1326EEG emotion recognitioncross-sensory emotion recognitionRiemannian feature extractionadversarial domain adaptation |
| spellingShingle | Chenguang Gao Hirotaka Uchitomi Yoshihiro Miyake Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation Brain Sciences EEG emotion recognition cross-sensory emotion recognition Riemannian feature extraction adversarial domain adaptation |
| title | Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation |
| title_full | Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation |
| title_fullStr | Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation |
| title_full_unstemmed | Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation |
| title_short | Cross-Sensory EEG Emotion Recognition with Filter Bank Riemannian Feature and Adversarial Domain Adaptation |
| title_sort | cross sensory eeg emotion recognition with filter bank riemannian feature and adversarial domain adaptation |
| topic | EEG emotion recognition cross-sensory emotion recognition Riemannian feature extraction adversarial domain adaptation |
| url | https://www.mdpi.com/2076-3425/13/9/1326 |
| work_keys_str_mv | AT chenguanggao crosssensoryeegemotionrecognitionwithfilterbankriemannianfeatureandadversarialdomainadaptation AT hirotakauchitomi crosssensoryeegemotionrecognitionwithfilterbankriemannianfeatureandadversarialdomainadaptation AT yoshihiromiyake crosssensoryeegemotionrecognitionwithfilterbankriemannianfeatureandadversarialdomainadaptation |