Experimental evidence of effective human–AI collaboration in medical decision-making
Abstract Artificial Intelligence (ai) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between mds and ai enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evide...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-09-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-18751-2 |
| _version_ | 1828114302738890752 |
|---|---|
| author | Carlo Reverberi Tommaso Rigon Aldo Solari Cesare Hassan Paolo Cherubini GI Genius CADx Study Group Andrea Cherubini |
| author_facet | Carlo Reverberi Tommaso Rigon Aldo Solari Cesare Hassan Paolo Cherubini GI Genius CADx Study Group Andrea Cherubini |
| author_sort | Carlo Reverberi |
| collection | DOAJ |
| description | Abstract Artificial Intelligence (ai) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between mds and ai enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human–ai collaboration. In this multicentric study, 21 endoscopists reviewed 504 videos of lesions prospectively acquired from real colonoscopies. They were asked to provide an optical diagnosis with and without the assistance of an ai support system. Endoscopists were influenced by ai ( $$\textsc {or}=3.05$$ O R = 3.05 ), but not erratically: they followed the ai advice more when it was correct ( $$\textsc {or}=3.48$$ O R = 3.48 ) than incorrect ( $$\textsc {or}=1.85$$ O R = 1.85 ). Endoscopists achieved this outcome through a weighted integration of their and the ai opinions, considering the case-by-case estimations of the two reliabilities. This Bayesian-like rational behavior allowed the human–ai hybrid team to outperform both agents taken alone. We discuss the features of the human–ai interaction that determined this favorable outcome. |
| first_indexed | 2024-04-11T12:22:17Z |
| format | Article |
| id | doaj.art-19f77efcf5f14a58b644e0ef75cb0cc0 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-11T12:22:17Z |
| publishDate | 2022-09-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-19f77efcf5f14a58b644e0ef75cb0cc02022-12-22T04:24:03ZengNature PortfolioScientific Reports2045-23222022-09-0112111010.1038/s41598-022-18751-2Experimental evidence of effective human–AI collaboration in medical decision-makingCarlo Reverberi0Tommaso Rigon1Aldo Solari2Cesare Hassan3Paolo Cherubini4GI Genius CADx Study GroupAndrea Cherubini5Department of Psychology, University of Milano-BicoccaDepartment of Economics, Management and Statistics, University of Milano-BicoccaMilan Center for Neuroscience, University of Milano-BicoccaDepartment of Biomedical Sciences, Humanitas UniversityDepartment of Psychology, University of Milano-BicoccaMilan Center for Neuroscience, University of Milano-BicoccaAbstract Artificial Intelligence (ai) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between mds and ai enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human–ai collaboration. In this multicentric study, 21 endoscopists reviewed 504 videos of lesions prospectively acquired from real colonoscopies. They were asked to provide an optical diagnosis with and without the assistance of an ai support system. Endoscopists were influenced by ai ( $$\textsc {or}=3.05$$ O R = 3.05 ), but not erratically: they followed the ai advice more when it was correct ( $$\textsc {or}=3.48$$ O R = 3.48 ) than incorrect ( $$\textsc {or}=1.85$$ O R = 1.85 ). Endoscopists achieved this outcome through a weighted integration of their and the ai opinions, considering the case-by-case estimations of the two reliabilities. This Bayesian-like rational behavior allowed the human–ai hybrid team to outperform both agents taken alone. We discuss the features of the human–ai interaction that determined this favorable outcome.https://doi.org/10.1038/s41598-022-18751-2 |
| spellingShingle | Carlo Reverberi Tommaso Rigon Aldo Solari Cesare Hassan Paolo Cherubini GI Genius CADx Study Group Andrea Cherubini Experimental evidence of effective human–AI collaboration in medical decision-making Scientific Reports |
| title | Experimental evidence of effective human–AI collaboration in medical decision-making |
| title_full | Experimental evidence of effective human–AI collaboration in medical decision-making |
| title_fullStr | Experimental evidence of effective human–AI collaboration in medical decision-making |
| title_full_unstemmed | Experimental evidence of effective human–AI collaboration in medical decision-making |
| title_short | Experimental evidence of effective human–AI collaboration in medical decision-making |
| title_sort | experimental evidence of effective human ai collaboration in medical decision making |
| url | https://doi.org/10.1038/s41598-022-18751-2 |
| work_keys_str_mv | AT carloreverberi experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking AT tommasorigon experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking AT aldosolari experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking AT cesarehassan experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking AT paolocherubini experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking AT gigeniuscadxstudygroup experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking AT andreacherubini experimentalevidenceofeffectivehumanaicollaborationinmedicaldecisionmaking |