The influence of dynamics and speech on understanding humanoid facial expressions
Human communication relies mostly on nonverbal signals expressed through body language. Facial expressions, in particular, convey emotional information that allows people involved in social interactions to mutually judge the emotional states and to adjust its behavior appropriately. First studies ai...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2018-07-01
|
| Series: | International Journal of Advanced Robotic Systems |
| Online Access: | https://doi.org/10.1177/1729881418783158 |
| _version_ | 1828794416450502656 |
|---|---|
| author | Nicole Lazzeri Daniele Mazzei Maher Ben Moussa Nadia Magnenat-Thalmann Danilo De Rossi |
| author_facet | Nicole Lazzeri Daniele Mazzei Maher Ben Moussa Nadia Magnenat-Thalmann Danilo De Rossi |
| author_sort | Nicole Lazzeri |
| collection | DOAJ |
| description | Human communication relies mostly on nonverbal signals expressed through body language. Facial expressions, in particular, convey emotional information that allows people involved in social interactions to mutually judge the emotional states and to adjust its behavior appropriately. First studies aimed at investigating the recognition of facial expressions were based on static stimuli. However, facial expressions are rarely static, especially in everyday social interactions. Therefore, it has been hypothesized that the dynamics inherent in a facial expression could be fundamental in understanding its meaning. In addition, it has been demonstrated that nonlinguistic and linguistic information can contribute to reinforce the meaning of a facial expression making it easier to be recognized. Nevertheless, few studies have been performed on realistic humanoid robots. This experimental work aimed at demonstrating the human-like expressive capability of a humanoid robot by examining whether the effect of motion and vocal content influenced the perception of its facial expressions. The first part of the experiment aimed at studying the recognition capability of two kinds of stimuli related to the six basic expressions (i.e. anger, disgust, fear, happiness, sadness, and surprise): static stimuli, that is, photographs, and dynamic stimuli, that is, video recordings. The second and third parts were focused on comparing the same six basic expressions performed by a virtual avatar and by a physical robot under three different conditions: (1) muted facial expressions, (2) facial expressions with nonlinguistic vocalizations, and (3) facial expressions with an emotionally neutral verbal sentence. The results show that static stimuli performed by a human being and by the robot were more ambiguous than the corresponding dynamic stimuli on which motion and vocalization were associated. This hypothesis has been also investigated with a 3-dimensional replica of the physical robot demonstrating that even in case of a virtual avatar, dynamic and vocalization improve the emotional conveying capability. |
| first_indexed | 2024-12-12T03:45:05Z |
| format | Article |
| id | doaj.art-6578693b3cbe485eaa7ce30372953d2e |
| institution | Directory Open Access Journal |
| issn | 1729-8814 |
| language | English |
| last_indexed | 2024-12-12T03:45:05Z |
| publishDate | 2018-07-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | International Journal of Advanced Robotic Systems |
| spelling | doaj.art-6578693b3cbe485eaa7ce30372953d2e2022-12-22T00:39:34ZengSAGE PublishingInternational Journal of Advanced Robotic Systems1729-88142018-07-011510.1177/1729881418783158The influence of dynamics and speech on understanding humanoid facial expressionsNicole Lazzeri0Daniele Mazzei1Maher Ben Moussa2Nadia Magnenat-Thalmann3Danilo De Rossi4 Research Center E. Piaggio, Faculty of Engineering, University of Pisa, Pisa, Italy Computer Science Department, University of Pisa, Pisa, Italy Computer Science Centre, University of Geneva, Geneva, Switzerland MIRALab, CUI, University of Geneva, Geneva, Switzerland Research Center E. Piaggio, Faculty of Engineering, University of Pisa, Pisa, ItalyHuman communication relies mostly on nonverbal signals expressed through body language. Facial expressions, in particular, convey emotional information that allows people involved in social interactions to mutually judge the emotional states and to adjust its behavior appropriately. First studies aimed at investigating the recognition of facial expressions were based on static stimuli. However, facial expressions are rarely static, especially in everyday social interactions. Therefore, it has been hypothesized that the dynamics inherent in a facial expression could be fundamental in understanding its meaning. In addition, it has been demonstrated that nonlinguistic and linguistic information can contribute to reinforce the meaning of a facial expression making it easier to be recognized. Nevertheless, few studies have been performed on realistic humanoid robots. This experimental work aimed at demonstrating the human-like expressive capability of a humanoid robot by examining whether the effect of motion and vocal content influenced the perception of its facial expressions. The first part of the experiment aimed at studying the recognition capability of two kinds of stimuli related to the six basic expressions (i.e. anger, disgust, fear, happiness, sadness, and surprise): static stimuli, that is, photographs, and dynamic stimuli, that is, video recordings. The second and third parts were focused on comparing the same six basic expressions performed by a virtual avatar and by a physical robot under three different conditions: (1) muted facial expressions, (2) facial expressions with nonlinguistic vocalizations, and (3) facial expressions with an emotionally neutral verbal sentence. The results show that static stimuli performed by a human being and by the robot were more ambiguous than the corresponding dynamic stimuli on which motion and vocalization were associated. This hypothesis has been also investigated with a 3-dimensional replica of the physical robot demonstrating that even in case of a virtual avatar, dynamic and vocalization improve the emotional conveying capability.https://doi.org/10.1177/1729881418783158 |
| spellingShingle | Nicole Lazzeri Daniele Mazzei Maher Ben Moussa Nadia Magnenat-Thalmann Danilo De Rossi The influence of dynamics and speech on understanding humanoid facial expressions International Journal of Advanced Robotic Systems |
| title | The influence of dynamics and speech on understanding humanoid facial expressions |
| title_full | The influence of dynamics and speech on understanding humanoid facial expressions |
| title_fullStr | The influence of dynamics and speech on understanding humanoid facial expressions |
| title_full_unstemmed | The influence of dynamics and speech on understanding humanoid facial expressions |
| title_short | The influence of dynamics and speech on understanding humanoid facial expressions |
| title_sort | influence of dynamics and speech on understanding humanoid facial expressions |
| url | https://doi.org/10.1177/1729881418783158 |
| work_keys_str_mv | AT nicolelazzeri theinfluenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT danielemazzei theinfluenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT maherbenmoussa theinfluenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT nadiamagnenatthalmann theinfluenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT daniloderossi theinfluenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT nicolelazzeri influenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT danielemazzei influenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT maherbenmoussa influenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT nadiamagnenatthalmann influenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions AT daniloderossi influenceofdynamicsandspeechonunderstandinghumanoidfacialexpressions |