Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork
Introduction: Collaboration in teams composed of both humans and automation has an interdependent nature, which demands calibrated trust among all the team members. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, autom...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-08-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frobt.2023.1143723/full |
| _version_ | 1827155541232713728 |
|---|---|
| author | Carolina Centeio Jorge Nikki H. Bouman Catholijn M. Jonker Catholijn M. Jonker Myrthe L. Tielman |
| author_facet | Carolina Centeio Jorge Nikki H. Bouman Catholijn M. Jonker Catholijn M. Jonker Myrthe L. Tielman |
| author_sort | Carolina Centeio Jorge |
| collection | DOAJ |
| description | Introduction: Collaboration in teams composed of both humans and automation has an interdependent nature, which demands calibrated trust among all the team members. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automation occasionally fails to do its job, which leads to a decrease in a human’s trust. Research has found interesting effects of such a reduction of trust on the human’s trustworthiness, i.e., human characteristics that make them more or less reliable. This paper investigates how automation failure in a human-automation collaborative scenario affects the human’s trust in the automation, as well as a human’s trustworthiness towards the automation.Methods: We present a 2 × 2 mixed design experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a “moving-out” scenario. During the experiment, we measure the participants’ trustworthiness, trust, and liking regarding the automation, both subjectively and objectively.Results: Our results show that automation failure negatively affects the human’s trustworthiness, as well as their trust in and liking of the automation.Discussion: Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams and improving human-automation teamwork. |
| first_indexed | 2024-03-12T13:40:37Z |
| format | Article |
| id | doaj.art-ce75a772763541c5a6ed3857de2f6129 |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2025-03-20T22:55:11Z |
| publishDate | 2023-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-ce75a772763541c5a6ed3857de2f61292024-08-05T07:24:38ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442023-08-011010.3389/frobt.2023.11437231143723Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamworkCarolina Centeio Jorge0Nikki H. Bouman1Catholijn M. Jonker2Catholijn M. Jonker3Myrthe L. Tielman4Interactive Intelligence, Intelligent Systems Department, Delft University of Technology, Delft, NetherlandsInteractive Intelligence, Intelligent Systems Department, Delft University of Technology, Delft, NetherlandsInteractive Intelligence, Intelligent Systems Department, Delft University of Technology, Delft, NetherlandsLeiden Institute of Advanced Computer Science (LIACS), University of Leiden, Leiden, NetherlandsInteractive Intelligence, Intelligent Systems Department, Delft University of Technology, Delft, NetherlandsIntroduction: Collaboration in teams composed of both humans and automation has an interdependent nature, which demands calibrated trust among all the team members. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automation occasionally fails to do its job, which leads to a decrease in a human’s trust. Research has found interesting effects of such a reduction of trust on the human’s trustworthiness, i.e., human characteristics that make them more or less reliable. This paper investigates how automation failure in a human-automation collaborative scenario affects the human’s trust in the automation, as well as a human’s trustworthiness towards the automation.Methods: We present a 2 × 2 mixed design experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a “moving-out” scenario. During the experiment, we measure the participants’ trustworthiness, trust, and liking regarding the automation, both subjectively and objectively.Results: Our results show that automation failure negatively affects the human’s trustworthiness, as well as their trust in and liking of the automation.Discussion: Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams and improving human-automation teamwork.https://www.frontiersin.org/articles/10.3389/frobt.2023.1143723/fullhuman-automation teamworkautomation failuremental modeltrusttrustworthinesshuman-agent collaboration |
| spellingShingle | Carolina Centeio Jorge Nikki H. Bouman Catholijn M. Jonker Catholijn M. Jonker Myrthe L. Tielman Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork Frontiers in Robotics and AI human-automation teamwork automation failure mental model trust trustworthiness human-agent collaboration |
| title | Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork |
| title_full | Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork |
| title_fullStr | Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork |
| title_full_unstemmed | Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork |
| title_short | Exploring the effect of automation failure on the human’s trustworthiness in human-agent teamwork |
| title_sort | exploring the effect of automation failure on the human s trustworthiness in human agent teamwork |
| topic | human-automation teamwork automation failure mental model trust trustworthiness human-agent collaboration |
| url | https://www.frontiersin.org/articles/10.3389/frobt.2023.1143723/full |
| work_keys_str_mv | AT carolinacenteiojorge exploringtheeffectofautomationfailureonthehumanstrustworthinessinhumanagentteamwork AT nikkihbouman exploringtheeffectofautomationfailureonthehumanstrustworthinessinhumanagentteamwork AT catholijnmjonker exploringtheeffectofautomationfailureonthehumanstrustworthinessinhumanagentteamwork AT catholijnmjonker exploringtheeffectofautomationfailureonthehumanstrustworthinessinhumanagentteamwork AT myrtheltielman exploringtheeffectofautomationfailureonthehumanstrustworthinessinhumanagentteamwork |