Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy
Abstract Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that...
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-03-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-08773-1 |
| _version_ | 1811335685074845696 |
|---|---|
| author | Lukas Buendgens Didem Cifci Narmin Ghaffari Laleh Marko van Treeck Maria T. Koenen Henning W. Zimmermann Till Herbold Thomas Joachim Lux Alexander Hann Christian Trautwein Jakob Nikolas Kather |
| author_facet | Lukas Buendgens Didem Cifci Narmin Ghaffari Laleh Marko van Treeck Maria T. Koenen Henning W. Zimmermann Till Herbold Thomas Joachim Lux Alexander Hann Christian Trautwein Jakob Nikolas Kather |
| author_sort | Lukas Buendgens |
| collection | DOAJ |
| description | Abstract Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N = 29,506 gastroscopy and N = 18,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities. |
| first_indexed | 2024-04-13T17:28:51Z |
| format | Article |
| id | doaj.art-7e5165609ff04e3d86e9ed2455175e99 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-13T17:28:51Z |
| publishDate | 2022-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-7e5165609ff04e3d86e9ed2455175e992022-12-22T02:37:40ZengNature PortfolioScientific Reports2045-23222022-03-0112111310.1038/s41598-022-08773-1Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopyLukas Buendgens0Didem Cifci1Narmin Ghaffari Laleh2Marko van Treeck3Maria T. Koenen4Henning W. Zimmermann5Till Herbold6Thomas Joachim Lux7Alexander Hann8Christian Trautwein9Jakob Nikolas Kather10Department of Medicine III, University Hospital RWTH AachenDepartment of Medicine III, University Hospital RWTH AachenDepartment of Medicine III, University Hospital RWTH AachenDepartment of Medicine III, University Hospital RWTH AachenDepartment of Medicine III, University Hospital RWTH AachenDepartment of Medicine III, University Hospital RWTH AachenDepartment of Visceral Surgery and Transplantation, University Hospital RWTH AachenInterventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital WürzburgInterventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital WürzburgDepartment of Medicine III, University Hospital RWTH AachenDepartment of Medicine III, University Hospital RWTH AachenAbstract Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N = 29,506 gastroscopy and N = 18,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities.https://doi.org/10.1038/s41598-022-08773-1 |
| spellingShingle | Lukas Buendgens Didem Cifci Narmin Ghaffari Laleh Marko van Treeck Maria T. Koenen Henning W. Zimmermann Till Herbold Thomas Joachim Lux Alexander Hann Christian Trautwein Jakob Nikolas Kather Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy Scientific Reports |
| title | Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy |
| title_full | Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy |
| title_fullStr | Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy |
| title_full_unstemmed | Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy |
| title_short | Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy |
| title_sort | weakly supervised end to end artificial intelligence in gastrointestinal endoscopy |
| url | https://doi.org/10.1038/s41598-022-08773-1 |
| work_keys_str_mv | AT lukasbuendgens weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT didemcifci weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT narminghaffarilaleh weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT markovantreeck weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT mariatkoenen weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT henningwzimmermann weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT tillherbold weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT thomasjoachimlux weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT alexanderhann weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT christiantrautwein weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy AT jakobnikolaskather weaklysupervisedendtoendartificialintelligenceingastrointestinalendoscopy |