Why do probabilistic clinical models fail to transport between sites
Abstract The rising popularity of artificial intelligence in healthcare is highlighting the problem that a computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. In this perspective, we argue that we should typically expect...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-03-01
|
| Series: | npj Digital Medicine |
| Online Access: | https://doi.org/10.1038/s41746-024-01037-4 |
| _version_ | 1797273105551327232 |
|---|---|
| author | Thomas A. Lasko Eric V. Strobl William W. Stead |
| author_facet | Thomas A. Lasko Eric V. Strobl William W. Stead |
| author_sort | Thomas A. Lasko |
| collection | DOAJ |
| description | Abstract The rising popularity of artificial intelligence in healthcare is highlighting the problem that a computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. In this perspective, we argue that we should typically expect this failure to transport, and we present common sources for it, divided into those under the control of the experimenter and those inherent to the clinical data-generating process. Of the inherent sources we look a little deeper into site-specific clinical practices that can affect the data distribution, and propose a potential solution intended to isolate the imprint of those practices on the data from the patterns of disease cause and effect that are the usual target of probabilistic clinical models. |
| first_indexed | 2024-03-07T14:39:47Z |
| format | Article |
| id | doaj.art-71b6d1f4d08f4b42be8fcc16a1f99077 |
| institution | Directory Open Access Journal |
| issn | 2398-6352 |
| language | English |
| last_indexed | 2024-03-07T14:39:47Z |
| publishDate | 2024-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Digital Medicine |
| spelling | doaj.art-71b6d1f4d08f4b42be8fcc16a1f990772024-03-05T20:27:11ZengNature Portfolionpj Digital Medicine2398-63522024-03-01711810.1038/s41746-024-01037-4Why do probabilistic clinical models fail to transport between sitesThomas A. Lasko0Eric V. Strobl1William W. Stead2Vanderbilt University Medical CenterVanderbilt University Medical CenterVanderbilt University Medical CenterAbstract The rising popularity of artificial intelligence in healthcare is highlighting the problem that a computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. In this perspective, we argue that we should typically expect this failure to transport, and we present common sources for it, divided into those under the control of the experimenter and those inherent to the clinical data-generating process. Of the inherent sources we look a little deeper into site-specific clinical practices that can affect the data distribution, and propose a potential solution intended to isolate the imprint of those practices on the data from the patterns of disease cause and effect that are the usual target of probabilistic clinical models.https://doi.org/10.1038/s41746-024-01037-4 |
| spellingShingle | Thomas A. Lasko Eric V. Strobl William W. Stead Why do probabilistic clinical models fail to transport between sites npj Digital Medicine |
| title | Why do probabilistic clinical models fail to transport between sites |
| title_full | Why do probabilistic clinical models fail to transport between sites |
| title_fullStr | Why do probabilistic clinical models fail to transport between sites |
| title_full_unstemmed | Why do probabilistic clinical models fail to transport between sites |
| title_short | Why do probabilistic clinical models fail to transport between sites |
| title_sort | why do probabilistic clinical models fail to transport between sites |
| url | https://doi.org/10.1038/s41746-024-01037-4 |
| work_keys_str_mv | AT thomasalasko whydoprobabilisticclinicalmodelsfailtotransportbetweensites AT ericvstrobl whydoprobabilisticclinicalmodelsfailtotransportbetweensites AT williamwstead whydoprobabilisticclinicalmodelsfailtotransportbetweensites |