Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces
With the continuous development of portable noninvasive human sensor technologies such as brain–computer interfaces (BCI), multimodal emotion recognition has attracted increasing attention in the area of affective computing. This paper primarily discusses the progress of research into multimodal emo...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-09-01
|
| Series: | Brain Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3425/10/10/687 |
| _version_ | 1797552257691025408 |
|---|---|
| author | Zhipeng He Zina Li Fuzhou Yang Lei Wang Jingcong Li Chengju Zhou Jiahui Pan |
| author_facet | Zhipeng He Zina Li Fuzhou Yang Lei Wang Jingcong Li Chengju Zhou Jiahui Pan |
| author_sort | Zhipeng He |
| collection | DOAJ |
| description | With the continuous development of portable noninvasive human sensor technologies such as brain–computer interfaces (BCI), multimodal emotion recognition has attracted increasing attention in the area of affective computing. This paper primarily discusses the progress of research into multimodal emotion recognition based on BCI and reviews three types of multimodal affective BCI (aBCI): aBCI based on a combination of behavior and brain signals, aBCI based on various hybrid neurophysiology modalities and aBCI based on heterogeneous sensory stimuli. For each type of aBCI, we further review several representative multimodal aBCI systems, including their design principles, paradigms, algorithms, experimental results and corresponding advantages. Finally, we identify several important issues and research directions for multimodal emotion recognition based on BCI. |
| first_indexed | 2024-03-10T15:57:27Z |
| format | Article |
| id | doaj.art-012298e556ac41e799a2d646e0b07787 |
| institution | Directory Open Access Journal |
| issn | 2076-3425 |
| language | English |
| last_indexed | 2024-03-10T15:57:27Z |
| publishDate | 2020-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Brain Sciences |
| spelling | doaj.art-012298e556ac41e799a2d646e0b077872023-11-20T15:29:45ZengMDPI AGBrain Sciences2076-34252020-09-01101068710.3390/brainsci10100687Advances in Multimodal Emotion Recognition Based on Brain–Computer InterfacesZhipeng He0Zina Li1Fuzhou Yang2Lei Wang3Jingcong Li4Chengju Zhou5Jiahui Pan6School of Software, South China Normal University, Foshan 528225, ChinaSchool of Computer, South China Normal University, Guangzhou 510641, ChinaSchool of Software, South China Normal University, Foshan 528225, ChinaSchool of Software, South China Normal University, Foshan 528225, ChinaSchool of Software, South China Normal University, Foshan 528225, ChinaSchool of Software, South China Normal University, Foshan 528225, ChinaSchool of Software, South China Normal University, Foshan 528225, ChinaWith the continuous development of portable noninvasive human sensor technologies such as brain–computer interfaces (BCI), multimodal emotion recognition has attracted increasing attention in the area of affective computing. This paper primarily discusses the progress of research into multimodal emotion recognition based on BCI and reviews three types of multimodal affective BCI (aBCI): aBCI based on a combination of behavior and brain signals, aBCI based on various hybrid neurophysiology modalities and aBCI based on heterogeneous sensory stimuli. For each type of aBCI, we further review several representative multimodal aBCI systems, including their design principles, paradigms, algorithms, experimental results and corresponding advantages. Finally, we identify several important issues and research directions for multimodal emotion recognition based on BCI.https://www.mdpi.com/2076-3425/10/10/687emotion recognitionmultimodal fusionbrain–computer interface (BCI)affective computing |
| spellingShingle | Zhipeng He Zina Li Fuzhou Yang Lei Wang Jingcong Li Chengju Zhou Jiahui Pan Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces Brain Sciences emotion recognition multimodal fusion brain–computer interface (BCI) affective computing |
| title | Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces |
| title_full | Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces |
| title_fullStr | Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces |
| title_full_unstemmed | Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces |
| title_short | Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces |
| title_sort | advances in multimodal emotion recognition based on brain computer interfaces |
| topic | emotion recognition multimodal fusion brain–computer interface (BCI) affective computing |
| url | https://www.mdpi.com/2076-3425/10/10/687 |
| work_keys_str_mv | AT zhipenghe advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces AT zinali advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces AT fuzhouyang advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces AT leiwang advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces AT jingcongli advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces AT chengjuzhou advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces AT jiahuipan advancesinmultimodalemotionrecognitionbasedonbraincomputerinterfaces |