Multimodal User Feedback During Adaptive Robot-Human Presentations
Feedback is an essential part of all communication, and agents communicating with humans must be able to both give and receive feedback in order to ensure mutual understanding. In this paper, we analyse multimodal feedback given by humans towards a robot that is presenting a piece of art in a shared...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-01-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2021.741148/full |
| _version_ | 1819098999398858752 |
|---|---|
| author | Agnes Axelsson Gabriel Skantze |
| author_facet | Agnes Axelsson Gabriel Skantze |
| author_sort | Agnes Axelsson |
| collection | DOAJ |
| description | Feedback is an essential part of all communication, and agents communicating with humans must be able to both give and receive feedback in order to ensure mutual understanding. In this paper, we analyse multimodal feedback given by humans towards a robot that is presenting a piece of art in a shared environment, similar to a museum setting. The data analysed contains both video and audio recordings of 28 participants, and the data has been richly annotated both in terms of multimodal cues (speech, gaze, head gestures, facial expressions, and body pose), as well as the polarity of any feedback (negative, positive, or neutral). We train statistical and machine learning models on the dataset, and find that random forest models and multinomial regression models perform well on predicting the polarity of the participants' reactions. An analysis of the different modalities shows that most information is found in the participants' speech and head gestures, while much less information is found in their facial expressions, body pose and gaze. An analysis of the timing of the feedback shows that most feedback is given when the robot makes pauses (and thereby invites feedback), but that the more exact timing of the feedback does not affect its meaning. |
| first_indexed | 2024-12-22T00:39:54Z |
| format | Article |
| id | doaj.art-b4f3e93dc8c64eafa1ace10ed685797c |
| institution | Directory Open Access Journal |
| issn | 2624-9898 |
| language | English |
| last_indexed | 2024-12-22T00:39:54Z |
| publishDate | 2022-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj.art-b4f3e93dc8c64eafa1ace10ed685797c2022-12-21T18:44:43ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982022-01-01310.3389/fcomp.2021.741148741148Multimodal User Feedback During Adaptive Robot-Human PresentationsAgnes AxelssonGabriel SkantzeFeedback is an essential part of all communication, and agents communicating with humans must be able to both give and receive feedback in order to ensure mutual understanding. In this paper, we analyse multimodal feedback given by humans towards a robot that is presenting a piece of art in a shared environment, similar to a museum setting. The data analysed contains both video and audio recordings of 28 participants, and the data has been richly annotated both in terms of multimodal cues (speech, gaze, head gestures, facial expressions, and body pose), as well as the polarity of any feedback (negative, positive, or neutral). We train statistical and machine learning models on the dataset, and find that random forest models and multinomial regression models perform well on predicting the polarity of the participants' reactions. An analysis of the different modalities shows that most information is found in the participants' speech and head gestures, while much less information is found in their facial expressions, body pose and gaze. An analysis of the timing of the feedback shows that most feedback is given when the robot makes pauses (and thereby invites feedback), but that the more exact timing of the feedback does not affect its meaning.https://www.frontiersin.org/articles/10.3389/fcomp.2021.741148/fullfeedbackpresentationagentrobotgroundingpolarity |
| spellingShingle | Agnes Axelsson Gabriel Skantze Multimodal User Feedback During Adaptive Robot-Human Presentations Frontiers in Computer Science feedback presentation agent robot grounding polarity |
| title | Multimodal User Feedback During Adaptive Robot-Human Presentations |
| title_full | Multimodal User Feedback During Adaptive Robot-Human Presentations |
| title_fullStr | Multimodal User Feedback During Adaptive Robot-Human Presentations |
| title_full_unstemmed | Multimodal User Feedback During Adaptive Robot-Human Presentations |
| title_short | Multimodal User Feedback During Adaptive Robot-Human Presentations |
| title_sort | multimodal user feedback during adaptive robot human presentations |
| topic | feedback presentation agent robot grounding polarity |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2021.741148/full |
| work_keys_str_mv | AT agnesaxelsson multimodaluserfeedbackduringadaptiverobothumanpresentations AT gabrielskantze multimodaluserfeedbackduringadaptiverobothumanpresentations |