Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals
With the development of cognitive science and brain science, brain-computer interface technology can use Electroencephalogram (EEG) signals to better represent the inner changes of emotions. In this paper, A video-induced emotional stimulation experimental paradigm was designed, and the EEG signals...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10109720/ |
| _version_ | 1797808485011816448 |
|---|---|
| author | Zhongli Bai Zeyu Li Zhiwei Li Yu Song Qiang Gao Zemin Mao |
| author_facet | Zhongli Bai Zeyu Li Zhiwei Li Yu Song Qiang Gao Zemin Mao |
| author_sort | Zhongli Bai |
| collection | DOAJ |
| description | With the development of cognitive science and brain science, brain-computer interface technology can use Electroencephalogram (EEG) signals to better represent the inner changes of emotions. In this paper, A video-induced emotional stimulation experimental paradigm was designed, and the EEG signals of 15 hearing-impaired subjects under three emotions (positive, neutral, and negative) were collected. Considering the flow diffusion properties of EEG signals, we used the diffusion effect based on horizontal representation and vertical representation forms to obtain the spatial domain features. After EEG preprocessing, the differential entropy feature (DE) in the frequency domain is extracted. The frequency domain features of 62 channels are delivered to two Bi-directional Long Short-Term Memory (BiLSTM) to obtain spatial domain features of horizontal and vertical representations respectively, and then two kinds of domain features are fused by the residual network. The attention mechanism is applied to effectively extract emotional representational information from the fused features. To solve the cross-subject problem of emotion recognition, the domain adaptation method is utilized, and a center alignment loss function is applied to increase the distance of inter-class and reduce the distance of intra-class. According to the experimental results, the average accuracies of 75.89% (subject- dependent) and 69.59% (cross-subject) are obtained. Moreover, the validation was also performed on the public dataset SEED, achieving average accuracies of 93.99% (subject-dependent) and 84.22% (cross-subject), respectively. |
| first_indexed | 2024-03-13T06:38:13Z |
| format | Article |
| id | doaj.art-86fb58be06c64f14b68063a9454a9f84 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T06:38:13Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-86fb58be06c64f14b68063a9454a9f842023-06-08T23:00:41ZengIEEEIEEE Access2169-35362023-01-0111550235503410.1109/ACCESS.2023.327097710109720Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG SignalsZhongli Bai0https://orcid.org/0000-0002-8036-3770Zeyu Li1Zhiwei Li2Yu Song3https://orcid.org/0000-0002-9295-7795Qiang Gao4https://orcid.org/0000-0001-9357-4967Zemin Mao5Tianjin Key Laboratory for Control Theory and Applications in Complicated Systems, School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin, ChinaTianjin Key Laboratory for Control Theory and Applications in Complicated Systems, School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin, ChinaTianjin Key Laboratory for Control Theory and Applications in Complicated Systems, School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin, ChinaTianjin Key Laboratory for Control Theory and Applications in Complicated Systems, School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin, ChinaTianjin Key Laboratory for Control Theory and Applications in Complicated Systems, TUT Maritime College, Tianjin University of Technology, Tianjin, ChinaTechnical College for the Deaf, Tianjin University of Technology, Tianjin, ChinaWith the development of cognitive science and brain science, brain-computer interface technology can use Electroencephalogram (EEG) signals to better represent the inner changes of emotions. In this paper, A video-induced emotional stimulation experimental paradigm was designed, and the EEG signals of 15 hearing-impaired subjects under three emotions (positive, neutral, and negative) were collected. Considering the flow diffusion properties of EEG signals, we used the diffusion effect based on horizontal representation and vertical representation forms to obtain the spatial domain features. After EEG preprocessing, the differential entropy feature (DE) in the frequency domain is extracted. The frequency domain features of 62 channels are delivered to two Bi-directional Long Short-Term Memory (BiLSTM) to obtain spatial domain features of horizontal and vertical representations respectively, and then two kinds of domain features are fused by the residual network. The attention mechanism is applied to effectively extract emotional representational information from the fused features. To solve the cross-subject problem of emotion recognition, the domain adaptation method is utilized, and a center alignment loss function is applied to increase the distance of inter-class and reduce the distance of intra-class. According to the experimental results, the average accuracies of 75.89% (subject- dependent) and 69.59% (cross-subject) are obtained. Moreover, the validation was also performed on the public dataset SEED, achieving average accuracies of 93.99% (subject-dependent) and 84.22% (cross-subject), respectively.https://ieeexplore.ieee.org/document/10109720/EEG signalsemotion recognitiondomain adaptationdeep learning |
| spellingShingle | Zhongli Bai Zeyu Li Zhiwei Li Yu Song Qiang Gao Zemin Mao Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals IEEE Access EEG signals emotion recognition domain adaptation deep learning |
| title | Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals |
| title_full | Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals |
| title_fullStr | Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals |
| title_full_unstemmed | Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals |
| title_short | Domain-Adaptive Emotion Recognition Based on Horizontal Vertical Flow Representation of EEG Signals |
| title_sort | domain adaptive emotion recognition based on horizontal vertical flow representation of eeg signals |
| topic | EEG signals emotion recognition domain adaptation deep learning |
| url | https://ieeexplore.ieee.org/document/10109720/ |
| work_keys_str_mv | AT zhonglibai domainadaptiveemotionrecognitionbasedonhorizontalverticalflowrepresentationofeegsignals AT zeyuli domainadaptiveemotionrecognitionbasedonhorizontalverticalflowrepresentationofeegsignals AT zhiweili domainadaptiveemotionrecognitionbasedonhorizontalverticalflowrepresentationofeegsignals AT yusong domainadaptiveemotionrecognitionbasedonhorizontalverticalflowrepresentationofeegsignals AT qianggao domainadaptiveemotionrecognitionbasedonhorizontalverticalflowrepresentationofeegsignals AT zeminmao domainadaptiveemotionrecognitionbasedonhorizontalverticalflowrepresentationofeegsignals |