Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order
Artificial intelligence (AI) is advancing across technology domains including healthcare, commerce, the economy, the environment, cybersecurity, transportation, etc. AI will transform healthcare systems, bringing profound changes to diagnosis, treatment, patient care, data, medicines, devices, etc....
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Systems |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-8954/12/2/47 |
| _version_ | 1797296961431273472 |
|---|---|
| author | Negin Moghadasi Rupa S. Valdez Misagh Piran Negar Moghaddasi Igor Linkov Thomas L. Polmateer Davis C. Loose James H. Lambert |
| author_facet | Negin Moghadasi Rupa S. Valdez Misagh Piran Negar Moghaddasi Igor Linkov Thomas L. Polmateer Davis C. Loose James H. Lambert |
| author_sort | Negin Moghadasi |
| collection | DOAJ |
| description | Artificial intelligence (AI) is advancing across technology domains including healthcare, commerce, the economy, the environment, cybersecurity, transportation, etc. AI will transform healthcare systems, bringing profound changes to diagnosis, treatment, patient care, data, medicines, devices, etc. However, AI in healthcare introduces entirely new categories of risk for assessment, management, and communication. For this topic, the framing of conventional risk and decision analyses is ongoing. This paper introduces a method to quantify risk as the disruption of the order of AI initiatives in healthcare systems, aiming to find the scenarios that are most and least disruptive to system order. This novel approach addresses scenarios that bring about a re-ordering of initiatives in each of the following three characteristic layers: purpose, structure, and function. In each layer, the following model elements are identified: 1. Typical research and development initiatives in healthcare. 2. The ordering criteria of the initiatives. 3. Emergent conditions and scenarios that could influence the ordering of the AI initiatives. This approach is a manifold accounting of the scenarios that could contribute to the risk associated with AI in healthcare. Recognizing the context-specific nature of risks and highlighting the role of human in the loop, this study identifies scenario <i>s.06</i>—non-interpretable AI and lack of human–AI communications—as the most disruptive across all three layers of healthcare systems. This finding suggests that AI transparency solutions primarily target domain experts, a reasonable inclination given the significance of “high-stakes” AI systems, particularly in healthcare. Future work should connect this approach with decision analysis and quantifying the value of information. Future work will explore the disruptions of system order in additional layers of the healthcare system, including the <i>environment</i>, <i>boundary</i>, <i>interconnections</i>, <i>workforce</i>, <i>facilities</i>, <i>supply chains</i>, and others. |
| first_indexed | 2024-03-07T22:12:24Z |
| format | Article |
| id | doaj.art-c3e8f0c0388947c6a1f2d710fe3e08a9 |
| institution | Directory Open Access Journal |
| issn | 2079-8954 |
| language | English |
| last_indexed | 2024-03-07T22:12:24Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Systems |
| spelling | doaj.art-c3e8f0c0388947c6a1f2d710fe3e08a92024-02-23T15:36:10ZengMDPI AGSystems2079-89542024-02-011224710.3390/systems12020047Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System OrderNegin Moghadasi0Rupa S. Valdez1Misagh Piran2Negar Moghaddasi3Igor Linkov4Thomas L. Polmateer5Davis C. Loose6James H. Lambert7Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22903, USADepartment of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22903, USADepartment of Radiology, Nuclear Medicine and Molecular Imaging, Heart and Diabetes Center North-Rhine Westphalia, Ruhr University of Bochum, 44801 Bochum, GermanyDepartment of Dentistry, Western University of Health Sciences, Pomona, CA 91766, USADepartment of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA 15213, USADepartment of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22903, USADepartment of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22903, USADepartment of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22903, USAArtificial intelligence (AI) is advancing across technology domains including healthcare, commerce, the economy, the environment, cybersecurity, transportation, etc. AI will transform healthcare systems, bringing profound changes to diagnosis, treatment, patient care, data, medicines, devices, etc. However, AI in healthcare introduces entirely new categories of risk for assessment, management, and communication. For this topic, the framing of conventional risk and decision analyses is ongoing. This paper introduces a method to quantify risk as the disruption of the order of AI initiatives in healthcare systems, aiming to find the scenarios that are most and least disruptive to system order. This novel approach addresses scenarios that bring about a re-ordering of initiatives in each of the following three characteristic layers: purpose, structure, and function. In each layer, the following model elements are identified: 1. Typical research and development initiatives in healthcare. 2. The ordering criteria of the initiatives. 3. Emergent conditions and scenarios that could influence the ordering of the AI initiatives. This approach is a manifold accounting of the scenarios that could contribute to the risk associated with AI in healthcare. Recognizing the context-specific nature of risks and highlighting the role of human in the loop, this study identifies scenario <i>s.06</i>—non-interpretable AI and lack of human–AI communications—as the most disruptive across all three layers of healthcare systems. This finding suggests that AI transparency solutions primarily target domain experts, a reasonable inclination given the significance of “high-stakes” AI systems, particularly in healthcare. Future work should connect this approach with decision analysis and quantifying the value of information. Future work will explore the disruptions of system order in additional layers of the healthcare system, including the <i>environment</i>, <i>boundary</i>, <i>interconnections</i>, <i>workforce</i>, <i>facilities</i>, <i>supply chains</i>, and others.https://www.mdpi.com/2079-8954/12/2/47risk managementrisk communicationinterpretable and explainable AIsystems engineeringscenario-based preferences |
| spellingShingle | Negin Moghadasi Rupa S. Valdez Misagh Piran Negar Moghaddasi Igor Linkov Thomas L. Polmateer Davis C. Loose James H. Lambert Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order Systems risk management risk communication interpretable and explainable AI systems engineering scenario-based preferences |
| title | Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order |
| title_full | Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order |
| title_fullStr | Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order |
| title_full_unstemmed | Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order |
| title_short | Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order |
| title_sort | risk analysis of artificial intelligence in medicine with a multilayer concept of system order |
| topic | risk management risk communication interpretable and explainable AI systems engineering scenario-based preferences |
| url | https://www.mdpi.com/2079-8954/12/2/47 |
| work_keys_str_mv | AT neginmoghadasi riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT rupasvaldez riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT misaghpiran riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT negarmoghaddasi riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT igorlinkov riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT thomaslpolmateer riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT daviscloose riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder AT jameshlambert riskanalysisofartificialintelligenceinmedicinewithamultilayerconceptofsystemorder |