The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations
Explainable artificial intelligence (XAI) has shown benefits in clinical decision support systems (CDSSs); however, it is still unclear to CDSS developers how to select an XAI method to optimize the advice-taking of healthcare practitioners. We performed a user study on healthcare practitioners base...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/20/10323 |
| _version_ | 1797475609601900544 |
|---|---|
| author | Yuhan Du Anna Markella Antoniadi Catherine McNestry Fionnuala M. McAuliffe Catherine Mooney |
| author_facet | Yuhan Du Anna Markella Antoniadi Catherine McNestry Fionnuala M. McAuliffe Catherine Mooney |
| author_sort | Yuhan Du |
| collection | DOAJ |
| description | Explainable artificial intelligence (XAI) has shown benefits in clinical decision support systems (CDSSs); however, it is still unclear to CDSS developers how to select an XAI method to optimize the advice-taking of healthcare practitioners. We performed a user study on healthcare practitioners based on a machine learning-based CDSS for the prediction of gestational diabetes mellitus to explore and compare two XAI methods: explanation by feature contribution and explanation by example. Participants were asked to make estimates for both correctly and incorrectly predicted cases to determine if there were any over-reliance or self-reliance issues. We examined the weight of advice and healthcare practitioners’ preferences. Our results based on statistical tests showed no significant difference between the two XAI methods regarding the advice-taking. The CDSS explained by either method had a substantial impact on the decision-making of healthcare practitioners; however, both methods may lead to over-reliance issues. We identified the inclination towards CDSS use as a key factor in the advice-taking from an explainable CDSS among obstetricians. Additionally, we found that different types of healthcare practitioners had differing preferences for explanations; therefore, we suggest that CDSS developers should select XAI methods according to their target users. |
| first_indexed | 2024-03-09T20:47:35Z |
| format | Article |
| id | doaj.art-5412dc7fc43b466f9640b9d03576c719 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T20:47:35Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-5412dc7fc43b466f9640b9d03576c7192023-11-23T22:42:48ZengMDPI AGApplied Sciences2076-34172022-10-0112201032310.3390/app122010323The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based ExplanationsYuhan Du0Anna Markella Antoniadi1Catherine McNestry2Fionnuala M. McAuliffe3Catherine Mooney4UCD School of Computer Science, University College Dublin, D04 V1W8 Dublin, IrelandUCD School of Computer Science, University College Dublin, D04 V1W8 Dublin, IrelandUCD Perinatal Research Centre, School of Medicine, University College Dublin, National Maternity Hospital, D02 NX40 Dublin, IrelandUCD Perinatal Research Centre, School of Medicine, University College Dublin, National Maternity Hospital, D02 NX40 Dublin, IrelandUCD School of Computer Science, University College Dublin, D04 V1W8 Dublin, IrelandExplainable artificial intelligence (XAI) has shown benefits in clinical decision support systems (CDSSs); however, it is still unclear to CDSS developers how to select an XAI method to optimize the advice-taking of healthcare practitioners. We performed a user study on healthcare practitioners based on a machine learning-based CDSS for the prediction of gestational diabetes mellitus to explore and compare two XAI methods: explanation by feature contribution and explanation by example. Participants were asked to make estimates for both correctly and incorrectly predicted cases to determine if there were any over-reliance or self-reliance issues. We examined the weight of advice and healthcare practitioners’ preferences. Our results based on statistical tests showed no significant difference between the two XAI methods regarding the advice-taking. The CDSS explained by either method had a substantial impact on the decision-making of healthcare practitioners; however, both methods may lead to over-reliance issues. We identified the inclination towards CDSS use as a key factor in the advice-taking from an explainable CDSS among obstetricians. Additionally, we found that different types of healthcare practitioners had differing preferences for explanations; therefore, we suggest that CDSS developers should select XAI methods according to their target users.https://www.mdpi.com/2076-3417/12/20/10323artificial intelligenceexplainable AIXAImachine learningclinical decision support systemCDSS |
| spellingShingle | Yuhan Du Anna Markella Antoniadi Catherine McNestry Fionnuala M. McAuliffe Catherine Mooney The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations Applied Sciences artificial intelligence explainable AI XAI machine learning clinical decision support system CDSS |
| title | The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations |
| title_full | The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations |
| title_fullStr | The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations |
| title_full_unstemmed | The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations |
| title_short | The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations |
| title_sort | role of xai in advice taking from a clinical decision support system a comparative user study of feature contribution based and example based explanations |
| topic | artificial intelligence explainable AI XAI machine learning clinical decision support system CDSS |
| url | https://www.mdpi.com/2076-3417/12/20/10323 |
| work_keys_str_mv | AT yuhandu theroleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT annamarkellaantoniadi theroleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT catherinemcnestry theroleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT fionnualammcauliffe theroleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT catherinemooney theroleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT yuhandu roleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT annamarkellaantoniadi roleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT catherinemcnestry roleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT fionnualammcauliffe roleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations AT catherinemooney roleofxaiinadvicetakingfromaclinicaldecisionsupportsystemacomparativeuserstudyoffeaturecontributionbasedandexamplebasedexplanations |