Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication
During the last decades, neurofeedback training for emotional self-regulation has received significant attention from scientific and clinical communities. Most studies have investigated emotions using functional magnetic resonance imaging (fMRI), including the real-time application in neurofeedback...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-08-01
|
| Series: | Frontiers in Human Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnhum.2021.711279/full |
| _version_ | 1819136353187659776 |
|---|---|
| author | Eleonora De Filippi Mara Wolter Bruno R. P. Melo Bruno R. P. Melo Carlos J. Tierra-Criollo Tiago Bortolini Gustavo Deco Gustavo Deco Gustavo Deco Gustavo Deco Jorge Moll Jorge Moll |
| author_facet | Eleonora De Filippi Mara Wolter Bruno R. P. Melo Bruno R. P. Melo Carlos J. Tierra-Criollo Tiago Bortolini Gustavo Deco Gustavo Deco Gustavo Deco Gustavo Deco Jorge Moll Jorge Moll |
| author_sort | Eleonora De Filippi |
| collection | DOAJ |
| description | During the last decades, neurofeedback training for emotional self-regulation has received significant attention from scientific and clinical communities. Most studies have investigated emotions using functional magnetic resonance imaging (fMRI), including the real-time application in neurofeedback training. However, the electroencephalogram (EEG) is a more suitable tool for therapeutic application. Our study aims at establishing a method to classify discrete complex emotions (e.g., tenderness and anguish) elicited through a near-immersive scenario that can be later used for EEG-neurofeedback. EEG-based affective computing studies have mainly focused on emotion classification based on dimensions, commonly using passive elicitation through single-modality stimuli. Here, we integrated both passive and active elicitation methods. We recorded electrophysiological data during emotion-evoking trials, combining emotional self-induction with a multimodal virtual environment. We extracted correlational and time-frequency features, including frontal-alpha asymmetry (FAA), using Complex Morlet Wavelet convolution. Thinking about future real-time applications, we performed within-subject classification using 1-s windows as samples and we applied trial-specific cross-validation. We opted for a traditional machine-learning classifier with low computational complexity and sufficient validation in online settings, the Support Vector Machine. Results of individual-based cross-validation using the whole feature sets showed considerable between-subject variability. The individual accuracies ranged from 59.2 to 92.9% using time-frequency/FAA and 62.4 to 92.4% using correlational features. We found that features of the temporal, occipital, and left-frontal channels were the most discriminative between the two emotions. Our results show that the suggested pipeline is suitable for individual-based classification of discrete emotions, paving the way for future personalized EEG-neurofeedback training. |
| first_indexed | 2024-12-22T10:33:38Z |
| format | Article |
| id | doaj.art-058677a490c64d619bfc4eed60c1a459 |
| institution | Directory Open Access Journal |
| issn | 1662-5161 |
| language | English |
| last_indexed | 2024-12-22T10:33:38Z |
| publishDate | 2021-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Human Neuroscience |
| spelling | doaj.art-058677a490c64d619bfc4eed60c1a4592022-12-21T18:29:15ZengFrontiers Media S.A.Frontiers in Human Neuroscience1662-51612021-08-011510.3389/fnhum.2021.711279711279Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic ImplicationEleonora De Filippi0Mara Wolter1Bruno R. P. Melo2Bruno R. P. Melo3Carlos J. Tierra-Criollo4Tiago Bortolini5Gustavo Deco6Gustavo Deco7Gustavo Deco8Gustavo Deco9Jorge Moll10Jorge Moll11Computational Neuroscience Group, Center for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, SpainCognitive Neuroscience and Neuroinformatics Unit, D'Or Institute for Research and Education (IDOR), Rio de Janeiro, BrazilCognitive Neuroscience and Neuroinformatics Unit, D'Or Institute for Research and Education (IDOR), Rio de Janeiro, BrazilBiomedical Engineering Program, Instituto Alberto Luiz Coimbra de Pós-Graduação e Pesquisa de Engenharia, Federal University of Rio de Janeiro, Rio de Janeiro, BrazilBiomedical Engineering Program, Instituto Alberto Luiz Coimbra de Pós-Graduação e Pesquisa de Engenharia, Federal University of Rio de Janeiro, Rio de Janeiro, BrazilCognitive Neuroscience and Neuroinformatics Unit, D'Or Institute for Research and Education (IDOR), Rio de Janeiro, BrazilComputational Neuroscience Group, Center for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, SpainInstitució Catalana de la Recerca i Estudis Avançats, Barcelona, SpainDepartment of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, GermanyTurner Institute for Brain and Mental Health, Monash University, Melbourne, VIC, AustraliaCognitive Neuroscience and Neuroinformatics Unit, D'Or Institute for Research and Education (IDOR), Rio de Janeiro, BrazilScients Institute, Palo Alto, CA, United StatesDuring the last decades, neurofeedback training for emotional self-regulation has received significant attention from scientific and clinical communities. Most studies have investigated emotions using functional magnetic resonance imaging (fMRI), including the real-time application in neurofeedback training. However, the electroencephalogram (EEG) is a more suitable tool for therapeutic application. Our study aims at establishing a method to classify discrete complex emotions (e.g., tenderness and anguish) elicited through a near-immersive scenario that can be later used for EEG-neurofeedback. EEG-based affective computing studies have mainly focused on emotion classification based on dimensions, commonly using passive elicitation through single-modality stimuli. Here, we integrated both passive and active elicitation methods. We recorded electrophysiological data during emotion-evoking trials, combining emotional self-induction with a multimodal virtual environment. We extracted correlational and time-frequency features, including frontal-alpha asymmetry (FAA), using Complex Morlet Wavelet convolution. Thinking about future real-time applications, we performed within-subject classification using 1-s windows as samples and we applied trial-specific cross-validation. We opted for a traditional machine-learning classifier with low computational complexity and sufficient validation in online settings, the Support Vector Machine. Results of individual-based cross-validation using the whole feature sets showed considerable between-subject variability. The individual accuracies ranged from 59.2 to 92.9% using time-frequency/FAA and 62.4 to 92.4% using correlational features. We found that features of the temporal, occipital, and left-frontal channels were the most discriminative between the two emotions. Our results show that the suggested pipeline is suitable for individual-based classification of discrete emotions, paving the way for future personalized EEG-neurofeedback training.https://www.frontiersin.org/articles/10.3389/fnhum.2021.711279/fullemotionselectroencephalographyclassificationmachine-learningneuro-feedbackmultimodal virtual scenario |
| spellingShingle | Eleonora De Filippi Mara Wolter Bruno R. P. Melo Bruno R. P. Melo Carlos J. Tierra-Criollo Tiago Bortolini Gustavo Deco Gustavo Deco Gustavo Deco Gustavo Deco Jorge Moll Jorge Moll Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication Frontiers in Human Neuroscience emotions electroencephalography classification machine-learning neuro-feedback multimodal virtual scenario |
| title | Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication |
| title_full | Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication |
| title_fullStr | Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication |
| title_full_unstemmed | Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication |
| title_short | Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication |
| title_sort | classification of complex emotions using eeg and virtual environment proof of concept and therapeutic implication |
| topic | emotions electroencephalography classification machine-learning neuro-feedback multimodal virtual scenario |
| url | https://www.frontiersin.org/articles/10.3389/fnhum.2021.711279/full |
| work_keys_str_mv | AT eleonoradefilippi classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT marawolter classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT brunorpmelo classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT brunorpmelo classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT carlosjtierracriollo classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT tiagobortolini classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT gustavodeco classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT gustavodeco classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT gustavodeco classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT gustavodeco classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT jorgemoll classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication AT jorgemoll classificationofcomplexemotionsusingeegandvirtualenvironmentproofofconceptandtherapeuticimplication |