Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
Machine learning is expected to fuel significant improvements in medical care. To ensure that fundamental principles such as beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency are respected, medical machine learning systems must be developed responsibly....
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9783196/ |
| _version_ | 1818219178933354496 |
|---|---|
| author | Eike Petersen Yannik Potdevin Esfandiar Mohammadi Stephan Zidowitz Sabrina Breyer Dirk Nowotka Sandra Henn Ludwig Pechmann Martin Leucker Philipp Rostalski Christian Herzog |
| author_facet | Eike Petersen Yannik Potdevin Esfandiar Mohammadi Stephan Zidowitz Sabrina Breyer Dirk Nowotka Sandra Henn Ludwig Pechmann Martin Leucker Philipp Rostalski Christian Herzog |
| author_sort | Eike Petersen |
| collection | DOAJ |
| description | Machine learning is expected to fuel significant improvements in medical care. To ensure that fundamental principles such as beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency are respected, medical machine learning systems must be developed responsibly. Many high-level declarations of ethical principles have been put forth for this purpose, but there is a severe lack of technical guidelines explicating the practical consequences for medical machine learning. Similarly, there is currently considerable uncertainty regarding the exact regulatory requirements placed upon medical machine learning systems. This survey provides an overview of the technical and procedural challenges involved in creating medical machine learning systems responsibly and in conformity with existing regulations, as well as possible solutions to address these challenges. First, a brief review of existing regulations affecting medical machine learning is provided, showing that properties such as safety, robustness, reliability, privacy, security, transparency, explainability, and nondiscrimination are all demanded <italic>already</italic> by existing law and regulations—albeit, in many cases, to an uncertain degree. Next, the key technical obstacles to achieving these desirable properties are discussed, as well as important techniques to overcome these obstacles in the medical context. We notice that distribution shift, spurious correlations, model underspecification, uncertainty quantification, and data scarcity represent severe challenges in the medical context. Promising solution approaches include the use of large and representative datasets and federated learning as a means to that end, the careful exploitation of domain knowledge, the use of inherently transparent models, comprehensive out-of-distribution model testing and verification, as well as algorithmic impact assessments. |
| first_indexed | 2024-12-12T07:35:32Z |
| format | Article |
| id | doaj.art-a749f14a5b3c4663b9a5629c8a0240db |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-12T07:35:32Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a749f14a5b3c4663b9a5629c8a0240db2022-12-22T00:32:55ZengIEEEIEEE Access2169-35362022-01-0110583755841810.1109/ACCESS.2022.31783829783196Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and SolutionsEike Petersen0https://orcid.org/0000-0003-0097-3868Yannik Potdevin1Esfandiar Mohammadi2https://orcid.org/0000-0002-2799-8128Stephan Zidowitz3https://orcid.org/0000-0001-8982-1830Sabrina Breyer4https://orcid.org/0000-0002-3853-3254Dirk Nowotka5https://orcid.org/0000-0002-5422-2229Sandra Henn6https://orcid.org/0000-0001-6943-9247Ludwig Pechmann7https://orcid.org/0000-0002-5390-2353Martin Leucker8https://orcid.org/0000-0002-3696-9222Philipp Rostalski9https://orcid.org/0000-0003-0326-168XChristian Herzog10https://orcid.org/0000-0003-2513-2563DTU Compute, Technical University of Denmark, Lyngby, DenmarkDepartment of Computer Science, Kiel University, Kiel, GermanyInstitute for IT Security (ITS), Universität zu Lübeck, Lübeck, GermanyFraunhofer Institute for Digital Medicine MEVIS, Bremen, GermanyInstitute for Electrical Engineering in Medicine (IME), Universität zu Lübeck, Lübeck, GermanyDepartment of Computer Science, Kiel University, Kiel, GermanyInstitute for Electrical Engineering in Medicine (IME), Universität zu Lübeck, Lübeck, GermanyUniTransferKlinik Lübeck GmbH, Lübeck, GermanyUniTransferKlinik Lübeck GmbH, Lübeck, GermanyInstitute for Electrical Engineering in Medicine (IME), Universität zu Lübeck, Lübeck, GermanyInstitute for Electrical Engineering in Medicine (IME), Universität zu Lübeck, Lübeck, GermanyMachine learning is expected to fuel significant improvements in medical care. To ensure that fundamental principles such as beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency are respected, medical machine learning systems must be developed responsibly. Many high-level declarations of ethical principles have been put forth for this purpose, but there is a severe lack of technical guidelines explicating the practical consequences for medical machine learning. Similarly, there is currently considerable uncertainty regarding the exact regulatory requirements placed upon medical machine learning systems. This survey provides an overview of the technical and procedural challenges involved in creating medical machine learning systems responsibly and in conformity with existing regulations, as well as possible solutions to address these challenges. First, a brief review of existing regulations affecting medical machine learning is provided, showing that properties such as safety, robustness, reliability, privacy, security, transparency, explainability, and nondiscrimination are all demanded <italic>already</italic> by existing law and regulations—albeit, in many cases, to an uncertain degree. Next, the key technical obstacles to achieving these desirable properties are discussed, as well as important techniques to overcome these obstacles in the medical context. We notice that distribution shift, spurious correlations, model underspecification, uncertainty quantification, and data scarcity represent severe challenges in the medical context. Promising solution approaches include the use of large and representative datasets and federated learning as a means to that end, the careful exploitation of domain knowledge, the use of inherently transparent models, comprehensive out-of-distribution model testing and verification, as well as algorithmic impact assessments.https://ieeexplore.ieee.org/document/9783196/Algorithmic fairnessethical machine learningexplainabilitymedical device regulationmedical machine learningprivacy |
| spellingShingle | Eike Petersen Yannik Potdevin Esfandiar Mohammadi Stephan Zidowitz Sabrina Breyer Dirk Nowotka Sandra Henn Ludwig Pechmann Martin Leucker Philipp Rostalski Christian Herzog Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions IEEE Access Algorithmic fairness ethical machine learning explainability medical device regulation medical machine learning privacy |
| title | Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions |
| title_full | Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions |
| title_fullStr | Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions |
| title_full_unstemmed | Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions |
| title_short | Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions |
| title_sort | responsible and regulatory conform machine learning for medicine a survey of challenges and solutions |
| topic | Algorithmic fairness ethical machine learning explainability medical device regulation medical machine learning privacy |
| url | https://ieeexplore.ieee.org/document/9783196/ |
| work_keys_str_mv | AT eikepetersen responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT yannikpotdevin responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT esfandiarmohammadi responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT stephanzidowitz responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT sabrinabreyer responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT dirknowotka responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT sandrahenn responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT ludwigpechmann responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT martinleucker responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT philipprostalski responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions AT christianherzog responsibleandregulatoryconformmachinelearningformedicineasurveyofchallengesandsolutions |