Human-centered explainability for life sciences, healthcare, and medical informatics
Summary: Rapid advances in artificial intelligence (AI) and availability of biological, medical, and healthcare data have enabled the development of a wide variety of models. Significant success has been achieved in a wide range of fields, such as genomics, protein folding, disease diagnosis, imagin...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-05-01
|
| Series: | Patterns |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666389922000782 |
| _version_ | 1828553871582035968 |
|---|---|
| author | Sanjoy Dey Prithwish Chakraborty Bum Chul Kwon Amit Dhurandhar Mohamed Ghalwash Fernando J. Suarez Saiz Kenney Ng Daby Sow Kush R. Varshney Pablo Meyer |
| author_facet | Sanjoy Dey Prithwish Chakraborty Bum Chul Kwon Amit Dhurandhar Mohamed Ghalwash Fernando J. Suarez Saiz Kenney Ng Daby Sow Kush R. Varshney Pablo Meyer |
| author_sort | Sanjoy Dey |
| collection | DOAJ |
| description | Summary: Rapid advances in artificial intelligence (AI) and availability of biological, medical, and healthcare data have enabled the development of a wide variety of models. Significant success has been achieved in a wide range of fields, such as genomics, protein folding, disease diagnosis, imaging, and clinical tasks. Although widely used, the inherent opacity of deep AI models has brought criticism from the research field and little adoption in clinical practice. Concurrently, there has been a significant amount of research focused on making such methods more interpretable, reviewed here, but inherent critiques of such explainability in AI (XAI), its requirements, and concerns with fairness/robustness have hampered their real-world adoption. We here discuss how user-driven XAI can be made more useful for different healthcare stakeholders through the definition of three key personas—data scientists, clinical researchers, and clinicians—and present an overview of how different XAI approaches can address their needs. For illustration, we also walk through several research and clinical examples that take advantage of XAI open-source tools, including those that help enhance the explanation of the results through visualization. This perspective thus aims to provide a guidance tool for developing explainability solutions for healthcare by empowering both subject matter experts, providing them with a survey of available tools, and explainability developers, by providing examples of how such methods can influence in practice adoption of solutions. The bigger picture: Adoption of AI tools in practical settings, such as for research/clinical tasks, has been hampered by a lack of transparency/interpretability of the models. After performing a review of different types of AI explainability (XAI) methods developed to better understand the predictions made by a model, we also develop a taxonomy to better classify the different approaches. We think that these XAI techniques are not sufficient to enhance practical implementations and illustrate via an example how user-driven XAI can be useful for different stakeholders in the healthcare domain. We identify and define three key personas involved in healthcare—data scientists, clinical researchers, and clinicians—and present an overview of the different approaches that can address their needs. The ultimate goal of adopting AI in medical practice and patient care goes beyond explainability and will need the development of extra layers of security and confidence, in particular regarding AI trustworthiness, as XAI transparent systems become prone to attacks that may reveal confidential information, and AI fairness, as systems developed and tested in diverse environments need to be expanded to real-world situations. |
| first_indexed | 2024-12-12T05:26:08Z |
| format | Article |
| id | doaj.art-ea48932b08a941d5a85c583bc912ac83 |
| institution | Directory Open Access Journal |
| issn | 2666-3899 |
| language | English |
| last_indexed | 2024-12-12T05:26:08Z |
| publishDate | 2022-05-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Patterns |
| spelling | doaj.art-ea48932b08a941d5a85c583bc912ac832022-12-22T00:36:28ZengElsevierPatterns2666-38992022-05-0135100493Human-centered explainability for life sciences, healthcare, and medical informaticsSanjoy Dey0Prithwish Chakraborty1Bum Chul Kwon2Amit Dhurandhar3Mohamed Ghalwash4Fernando J. Suarez Saiz5Kenney Ng6Daby Sow7Kush R. Varshney8Pablo Meyer9Center for Computational Health, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USACenter for Computational Health, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USACenter for Computational Health, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USAIBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USACenter for Computational Health, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA; Ain Shams University, Cairo, EgyptIBM Watson Health, New York, NY 10017, USACenter for Computational Health, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USAIBM Research Security and Compliance, AI Industries, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USAIBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USACenter for Computational Health, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA; Corresponding authorSummary: Rapid advances in artificial intelligence (AI) and availability of biological, medical, and healthcare data have enabled the development of a wide variety of models. Significant success has been achieved in a wide range of fields, such as genomics, protein folding, disease diagnosis, imaging, and clinical tasks. Although widely used, the inherent opacity of deep AI models has brought criticism from the research field and little adoption in clinical practice. Concurrently, there has been a significant amount of research focused on making such methods more interpretable, reviewed here, but inherent critiques of such explainability in AI (XAI), its requirements, and concerns with fairness/robustness have hampered their real-world adoption. We here discuss how user-driven XAI can be made more useful for different healthcare stakeholders through the definition of three key personas—data scientists, clinical researchers, and clinicians—and present an overview of how different XAI approaches can address their needs. For illustration, we also walk through several research and clinical examples that take advantage of XAI open-source tools, including those that help enhance the explanation of the results through visualization. This perspective thus aims to provide a guidance tool for developing explainability solutions for healthcare by empowering both subject matter experts, providing them with a survey of available tools, and explainability developers, by providing examples of how such methods can influence in practice adoption of solutions. The bigger picture: Adoption of AI tools in practical settings, such as for research/clinical tasks, has been hampered by a lack of transparency/interpretability of the models. After performing a review of different types of AI explainability (XAI) methods developed to better understand the predictions made by a model, we also develop a taxonomy to better classify the different approaches. We think that these XAI techniques are not sufficient to enhance practical implementations and illustrate via an example how user-driven XAI can be useful for different stakeholders in the healthcare domain. We identify and define three key personas involved in healthcare—data scientists, clinical researchers, and clinicians—and present an overview of the different approaches that can address their needs. The ultimate goal of adopting AI in medical practice and patient care goes beyond explainability and will need the development of extra layers of security and confidence, in particular regarding AI trustworthiness, as XAI transparent systems become prone to attacks that may reveal confidential information, and AI fairness, as systems developed and tested in diverse environments need to be expanded to real-world situations.http://www.sciencedirect.com/science/article/pii/S2666389922000782DSML5: Mainstream: Data science output is well understood and (nearly) universally adopted |
| spellingShingle | Sanjoy Dey Prithwish Chakraborty Bum Chul Kwon Amit Dhurandhar Mohamed Ghalwash Fernando J. Suarez Saiz Kenney Ng Daby Sow Kush R. Varshney Pablo Meyer Human-centered explainability for life sciences, healthcare, and medical informatics Patterns DSML5: Mainstream: Data science output is well understood and (nearly) universally adopted |
| title | Human-centered explainability for life sciences, healthcare, and medical informatics |
| title_full | Human-centered explainability for life sciences, healthcare, and medical informatics |
| title_fullStr | Human-centered explainability for life sciences, healthcare, and medical informatics |
| title_full_unstemmed | Human-centered explainability for life sciences, healthcare, and medical informatics |
| title_short | Human-centered explainability for life sciences, healthcare, and medical informatics |
| title_sort | human centered explainability for life sciences healthcare and medical informatics |
| topic | DSML5: Mainstream: Data science output is well understood and (nearly) universally adopted |
| url | http://www.sciencedirect.com/science/article/pii/S2666389922000782 |
| work_keys_str_mv | AT sanjoydey humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT prithwishchakraborty humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT bumchulkwon humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT amitdhurandhar humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT mohamedghalwash humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT fernandojsuarezsaiz humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT kenneyng humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT dabysow humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT kushrvarshney humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics AT pablomeyer humancenteredexplainabilityforlifescienceshealthcareandmedicalinformatics |