Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study
Emotion recognition, a rapidly evolving domain in digital health, has witnessed significant transformations with the advent of personalized approaches and advanced machine learning (ML) techniques. These advancements have shifted the focus from traditional, generalized models to more individual-cent...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/4/1337 |
| _version_ | 1797299017943613440 |
|---|---|
| author | Ali Kargarandehkordi Matti Kaisti Peter Washington |
| author_facet | Ali Kargarandehkordi Matti Kaisti Peter Washington |
| author_sort | Ali Kargarandehkordi |
| collection | DOAJ |
| description | Emotion recognition, a rapidly evolving domain in digital health, has witnessed significant transformations with the advent of personalized approaches and advanced machine learning (ML) techniques. These advancements have shifted the focus from traditional, generalized models to more individual-centric methodologies, underscoring the importance of understanding and catering to the unique emotional expressions of individuals. Our study delves into the concept of model personalization in emotion recognition, moving away from the one-size-fits-all approach. We conducted a series of experiments using the Emognition dataset, comprising physiological and video data of human subjects expressing various emotions, to investigate this personalized approach to affective computing. For the 10 individuals in the dataset with a sufficient representation of at least two ground truth emotion labels, we trained a personalized version of three classical ML models (k-nearest neighbors, random forests, and a dense neural network) on a set of 51 features extracted from each video frame. We ensured that all the frames used to train the models occurred earlier in the video than the frames used to test the model. We measured the importance of each facial feature for all the personalized models and observed differing ranked lists of the top features across the subjects, highlighting the need for model personalization. We then compared the personalized models against a generalized model trained using data from all 10 subjects. The mean F1 scores for the personalized models, specifically for the k-nearest neighbors, random forest, and dense neural network, were 90.48%, 92.66%, and 86.40%, respectively. In contrast, the mean F1 scores for the generic models, using the same ML techniques, were 88.55%, 91.78% and 80.42%, respectively, when trained on data from various human subjects and evaluated using the same test set. The personalized models outperformed the generalized models for 7 out of the 10 subjects. The PCA analyses on the remaining three subjects revealed relatively little facial configuration differences across the emotion labels within each subject, suggesting that personalized ML will fail when the variation among data points within a subject’s data is too low. This preliminary feasibility study demonstrates the potential as well as the ongoing challenges with implementing personalized models which predict highly subjective outcomes like emotion. |
| first_indexed | 2024-03-07T22:44:29Z |
| format | Article |
| id | doaj.art-58ce47ea48a7498e8368a40998232633 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-07T22:44:29Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-58ce47ea48a7498e8368a409982326332024-02-23T15:05:41ZengMDPI AGApplied Sciences2076-34172024-02-01144133710.3390/app14041337Personalization of Affective Models Using Classical Machine Learning: A Feasibility StudyAli Kargarandehkordi0Matti Kaisti1Peter Washington2Information and Computer Sciences Department, University of Hawai’i at Manoa, Honolulu, HI 96822, USADepartment of Computing, University of Turku, 20014 Turku, FinlandInformation and Computer Sciences Department, University of Hawai’i at Manoa, Honolulu, HI 96822, USAEmotion recognition, a rapidly evolving domain in digital health, has witnessed significant transformations with the advent of personalized approaches and advanced machine learning (ML) techniques. These advancements have shifted the focus from traditional, generalized models to more individual-centric methodologies, underscoring the importance of understanding and catering to the unique emotional expressions of individuals. Our study delves into the concept of model personalization in emotion recognition, moving away from the one-size-fits-all approach. We conducted a series of experiments using the Emognition dataset, comprising physiological and video data of human subjects expressing various emotions, to investigate this personalized approach to affective computing. For the 10 individuals in the dataset with a sufficient representation of at least two ground truth emotion labels, we trained a personalized version of three classical ML models (k-nearest neighbors, random forests, and a dense neural network) on a set of 51 features extracted from each video frame. We ensured that all the frames used to train the models occurred earlier in the video than the frames used to test the model. We measured the importance of each facial feature for all the personalized models and observed differing ranked lists of the top features across the subjects, highlighting the need for model personalization. We then compared the personalized models against a generalized model trained using data from all 10 subjects. The mean F1 scores for the personalized models, specifically for the k-nearest neighbors, random forest, and dense neural network, were 90.48%, 92.66%, and 86.40%, respectively. In contrast, the mean F1 scores for the generic models, using the same ML techniques, were 88.55%, 91.78% and 80.42%, respectively, when trained on data from various human subjects and evaluated using the same test set. The personalized models outperformed the generalized models for 7 out of the 10 subjects. The PCA analyses on the remaining three subjects revealed relatively little facial configuration differences across the emotion labels within each subject, suggesting that personalized ML will fail when the variation among data points within a subject’s data is too low. This preliminary feasibility study demonstrates the potential as well as the ongoing challenges with implementing personalized models which predict highly subjective outcomes like emotion.https://www.mdpi.com/2076-3417/14/4/1337personalized MLaffective computingASDdigital phenotypingemotiongeneric |
| spellingShingle | Ali Kargarandehkordi Matti Kaisti Peter Washington Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study Applied Sciences personalized ML affective computing ASD digital phenotyping emotion generic |
| title | Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study |
| title_full | Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study |
| title_fullStr | Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study |
| title_full_unstemmed | Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study |
| title_short | Personalization of Affective Models Using Classical Machine Learning: A Feasibility Study |
| title_sort | personalization of affective models using classical machine learning a feasibility study |
| topic | personalized ML affective computing ASD digital phenotyping emotion generic |
| url | https://www.mdpi.com/2076-3417/14/4/1337 |
| work_keys_str_mv | AT alikargarandehkordi personalizationofaffectivemodelsusingclassicalmachinelearningafeasibilitystudy AT mattikaisti personalizationofaffectivemodelsusingclassicalmachinelearningafeasibilitystudy AT peterwashington personalizationofaffectivemodelsusingclassicalmachinelearningafeasibilitystudy |