Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals
Emotion classification using physiological signals is a promising approach that is likely to become the most prevalent method. Bio-signals such as those derived from Electrocardiograms (ECGs) and the Galvanic Skin Response (GSR) are more reliable than facial and voice recognition signals because the...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Sci |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2413-4155/6/1/10 |
| _version_ | 1827305043681869824 |
|---|---|
| author | Amita Dessai Hassanali Virani |
| author_facet | Amita Dessai Hassanali Virani |
| author_sort | Amita Dessai |
| collection | DOAJ |
| description | Emotion classification using physiological signals is a promising approach that is likely to become the most prevalent method. Bio-signals such as those derived from Electrocardiograms (ECGs) and the Galvanic Skin Response (GSR) are more reliable than facial and voice recognition signals because they are not influenced by the participant’s subjective perception. However, the precision of emotion classification with ECG and GSR signals is not satisfactory, and new methods need to be developed to improve it. In addition, the fusion of the time and frequency features of ECG and GSR signals should be explored to increase classification accuracy. Therefore, we propose a novel technique for emotion classification that exploits the early fusion of ECG and GSR features extracted from data in the AMIGOS database. To validate the performance of the model, we used various machine learning classifiers, such as Support Vector Machine (SVM), Decision Tree, Random Forest (RF), and K-Nearest Neighbor (KNN) classifiers. The KNN classifier gives the highest accuracy for Valence and Arousal, with 69% and 70% for ECG and 96% and 94% for GSR, respectively. The mutual information technique of feature selection and KNN for classification outperformed the performance of other classifiers. Interestingly, the classification accuracy for the GSR was higher than for the ECG, indicating that the GSR is the preferred modality for emotion detection. Moreover, the fusion of features significantly enhances the accuracy of classification in comparison to the ECG. Overall, our findings demonstrate that the proposed model based on the multiple modalities is suitable for classifying emotions. |
| first_indexed | 2024-04-24T17:50:31Z |
| format | Article |
| id | doaj.art-61d67fffbfed48fa8532d44909f8a3cd |
| institution | Directory Open Access Journal |
| issn | 2413-4155 |
| language | English |
| last_indexed | 2024-04-24T17:50:31Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sci |
| spelling | doaj.art-61d67fffbfed48fa8532d44909f8a3cd2024-03-27T14:03:23ZengMDPI AGSci2413-41552024-02-01611010.3390/sci6010010Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response SignalsAmita Dessai0Hassanali Virani1Goa College of Engineering, Goa University, Ponda 403401, IndiaGoa College of Engineering, Goa University, Ponda 403401, IndiaEmotion classification using physiological signals is a promising approach that is likely to become the most prevalent method. Bio-signals such as those derived from Electrocardiograms (ECGs) and the Galvanic Skin Response (GSR) are more reliable than facial and voice recognition signals because they are not influenced by the participant’s subjective perception. However, the precision of emotion classification with ECG and GSR signals is not satisfactory, and new methods need to be developed to improve it. In addition, the fusion of the time and frequency features of ECG and GSR signals should be explored to increase classification accuracy. Therefore, we propose a novel technique for emotion classification that exploits the early fusion of ECG and GSR features extracted from data in the AMIGOS database. To validate the performance of the model, we used various machine learning classifiers, such as Support Vector Machine (SVM), Decision Tree, Random Forest (RF), and K-Nearest Neighbor (KNN) classifiers. The KNN classifier gives the highest accuracy for Valence and Arousal, with 69% and 70% for ECG and 96% and 94% for GSR, respectively. The mutual information technique of feature selection and KNN for classification outperformed the performance of other classifiers. Interestingly, the classification accuracy for the GSR was higher than for the ECG, indicating that the GSR is the preferred modality for emotion detection. Moreover, the fusion of features significantly enhances the accuracy of classification in comparison to the ECG. Overall, our findings demonstrate that the proposed model based on the multiple modalities is suitable for classifying emotions.https://www.mdpi.com/2413-4155/6/1/10emotionAMIGOSECGGSRmutual informationKNN |
| spellingShingle | Amita Dessai Hassanali Virani Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals Sci emotion AMIGOS ECG GSR mutual information KNN |
| title | Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals |
| title_full | Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals |
| title_fullStr | Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals |
| title_full_unstemmed | Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals |
| title_short | Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals |
| title_sort | multimodal and multidomain feature fusion for emotion classification based on electrocardiogram and galvanic skin response signals |
| topic | emotion AMIGOS ECG GSR mutual information KNN |
| url | https://www.mdpi.com/2413-4155/6/1/10 |
| work_keys_str_mv | AT amitadessai multimodalandmultidomainfeaturefusionforemotionclassificationbasedonelectrocardiogramandgalvanicskinresponsesignals AT hassanalivirani multimodalandmultidomainfeaturefusionforemotionclassificationbasedonelectrocardiogramandgalvanicskinresponsesignals |