Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition
Can we really “read the mind in the eyes”? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of th...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Future Internet |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-5903/14/1/5 |
| _version_ | 1797493941037170688 |
|---|---|
| author | Peter A. Gloor Andrea Fronzetti Colladon Erkin Altuntas Cengiz Cetinkaya Maximilian F. Kaiser Lukas Ripperger Tim Schaefer |
| author_facet | Peter A. Gloor Andrea Fronzetti Colladon Erkin Altuntas Cengiz Cetinkaya Maximilian F. Kaiser Lukas Ripperger Tim Schaefer |
| author_sort | Peter A. Gloor |
| collection | DOAJ |
| description | Can we really “read the mind in the eyes”? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual’s face through facial emotion recognition (FER) while watching a series of 15 short videos of different genres. To calibrate the system, we invited 85 people to watch the videos, while their emotional responses were analyzed through their facial expression. At the same time, these individuals also took four well-validated surveys of personality characteristics and moral values: the revised NEO FFI personality inventory, the Haidt moral foundations test, the Schwartz personal value system, and the domain-specific risk-taking scale (DOSPERT). We found that personality characteristics and moral values of an individual can be predicted through their emotional response to the videos as shown in their face, with an accuracy of up to 86% using gradient-boosted trees. We also found that different personality characteristics are better predicted by different videos, in other words, there is no single video that will provide accurate predictions for all personality characteristics, but it is the response to the mix of different videos that allows for accurate prediction. |
| first_indexed | 2024-03-10T01:27:12Z |
| format | Article |
| id | doaj.art-d71c6fc0485b4bc1a2018fc25cc4cfae |
| institution | Directory Open Access Journal |
| issn | 1999-5903 |
| language | English |
| last_indexed | 2024-03-10T01:27:12Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Future Internet |
| spelling | doaj.art-d71c6fc0485b4bc1a2018fc25cc4cfae2023-11-23T13:49:16ZengMDPI AGFuture Internet1999-59032021-12-01141510.3390/fi14010005Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion RecognitionPeter A. Gloor0Andrea Fronzetti Colladon1Erkin Altuntas2Cengiz Cetinkaya3Maximilian F. Kaiser4Lukas Ripperger5Tim Schaefer6MIT Center for Collective Intelligence, Cambridge, MA 02142, USADepartment of Engineering, University of Perugia, 06123 Perugia, ItalyGalaxyadvisors AG, 5000 Aarau, SwitzerlandDepartment of Data Science, Lucerne University of Applied Sciences and Arts, 6002 Lucerne, SwitzerlandDepartment of Information Systems, University of Cologne, 50923 Cologne, GermanyDepartment of Information Systems, University of Cologne, 50923 Cologne, GermanyDepartment of Information Systems, University of Cologne, 50923 Cologne, GermanyCan we really “read the mind in the eyes”? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual’s face through facial emotion recognition (FER) while watching a series of 15 short videos of different genres. To calibrate the system, we invited 85 people to watch the videos, while their emotional responses were analyzed through their facial expression. At the same time, these individuals also took four well-validated surveys of personality characteristics and moral values: the revised NEO FFI personality inventory, the Haidt moral foundations test, the Schwartz personal value system, and the domain-specific risk-taking scale (DOSPERT). We found that personality characteristics and moral values of an individual can be predicted through their emotional response to the videos as shown in their face, with an accuracy of up to 86% using gradient-boosted trees. We also found that different personality characteristics are better predicted by different videos, in other words, there is no single video that will provide accurate predictions for all personality characteristics, but it is the response to the mix of different videos that allows for accurate prediction.https://www.mdpi.com/1999-5903/14/1/5artificial intelligencefacial emotion recognitionpersonalitymoral valuesrisk-takingforecasting |
| spellingShingle | Peter A. Gloor Andrea Fronzetti Colladon Erkin Altuntas Cengiz Cetinkaya Maximilian F. Kaiser Lukas Ripperger Tim Schaefer Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition Future Internet artificial intelligence facial emotion recognition personality moral values risk-taking forecasting |
| title | Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition |
| title_full | Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition |
| title_fullStr | Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition |
| title_full_unstemmed | Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition |
| title_short | Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition |
| title_sort | your face mirrors your deepest beliefs predicting personality and morals through facial emotion recognition |
| topic | artificial intelligence facial emotion recognition personality moral values risk-taking forecasting |
| url | https://www.mdpi.com/1999-5903/14/1/5 |
| work_keys_str_mv | AT peteragloor yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition AT andreafronzetticolladon yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition AT erkinaltuntas yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition AT cengizcetinkaya yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition AT maximilianfkaiser yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition AT lukasripperger yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition AT timschaefer yourfacemirrorsyourdeepestbeliefspredictingpersonalityandmoralsthroughfacialemotionrecognition |