Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays
Abstract The AI-Rad Companion Chest X-ray (AI-Rad, Siemens Healthineers) is an artificial-intelligence based application for the analysis of chest X-rays. The purpose of the present study is to evaluate the performance of the AI-Rad. In total, 499 radiographs were retrospectively included. Radiograp...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-03-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-30521-2 |
| _version_ | 1797864808266072064 |
|---|---|
| author | Julius Henning Niehoff Jana Kalaitzidis Jan Robert Kroeger Denise Schoenbeck Jan Borggrefe Arwed Elias Michael |
| author_facet | Julius Henning Niehoff Jana Kalaitzidis Jan Robert Kroeger Denise Schoenbeck Jan Borggrefe Arwed Elias Michael |
| author_sort | Julius Henning Niehoff |
| collection | DOAJ |
| description | Abstract The AI-Rad Companion Chest X-ray (AI-Rad, Siemens Healthineers) is an artificial-intelligence based application for the analysis of chest X-rays. The purpose of the present study is to evaluate the performance of the AI-Rad. In total, 499 radiographs were retrospectively included. Radiographs were independently evaluated by radiologists and the AI-Rad. Findings indicated by the AI-Rad and findings described in the written report (WR) were compared to the findings of a ground truth reading (consensus decision of two radiologists after assessing additional radiographs and CT scans). The AI-Rad can offer superior sensitivity for the detection of lung lesions (0.83 versus 0.52), consolidations (0.88 versus 0.78) and atelectasis (0.54 versus 0.43) compared to the WR. However, the superior sensitivity is accompanied by higher false-detection-rates. The sensitivity of the AI-Rad for the detection of pleural effusions is lower compared to the WR (0.74 versus 0.88). The negative-predictive-values (NPV) of the AI-Rad for the detection of all pre-defined findings are on a high level and comparable to the WR. The seemingly advantageous high sensitivity of the AI-Rad is partially offset by the disadvantage of a high false-detection-rate. At the current stage of development, therefore, the high NPVs may be the greatest benefit of the AI-Rad giving radiologists the possibility to re-insure their own negative search for pathologies and thus boosting their confidence in their reports. |
| first_indexed | 2024-04-09T22:58:04Z |
| format | Article |
| id | doaj.art-1a72cf5c41234b85b5c6c978937447d0 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-09T22:58:04Z |
| publishDate | 2023-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-1a72cf5c41234b85b5c6c978937447d02023-03-22T11:08:04ZengNature PortfolioScientific Reports2045-23222023-03-0113111110.1038/s41598-023-30521-2Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-raysJulius Henning Niehoff0Jana Kalaitzidis1Jan Robert Kroeger2Denise Schoenbeck3Jan Borggrefe4Arwed Elias Michael5Department of Radiology, Neuroradiology and Nuclear Medicine, Johannes Wesling University Hospital, Ruhr University BochumDepartment of Radiology, Neuroradiology and Nuclear Medicine, Johannes Wesling University Hospital, Ruhr University BochumDepartment of Radiology, Neuroradiology and Nuclear Medicine, Johannes Wesling University Hospital, Ruhr University BochumDepartment of Radiology, Neuroradiology and Nuclear Medicine, Johannes Wesling University Hospital, Ruhr University BochumDepartment of Radiology, Neuroradiology and Nuclear Medicine, Johannes Wesling University Hospital, Ruhr University BochumDepartment of Radiology, Neuroradiology and Nuclear Medicine, Johannes Wesling University Hospital, Ruhr University BochumAbstract The AI-Rad Companion Chest X-ray (AI-Rad, Siemens Healthineers) is an artificial-intelligence based application for the analysis of chest X-rays. The purpose of the present study is to evaluate the performance of the AI-Rad. In total, 499 radiographs were retrospectively included. Radiographs were independently evaluated by radiologists and the AI-Rad. Findings indicated by the AI-Rad and findings described in the written report (WR) were compared to the findings of a ground truth reading (consensus decision of two radiologists after assessing additional radiographs and CT scans). The AI-Rad can offer superior sensitivity for the detection of lung lesions (0.83 versus 0.52), consolidations (0.88 versus 0.78) and atelectasis (0.54 versus 0.43) compared to the WR. However, the superior sensitivity is accompanied by higher false-detection-rates. The sensitivity of the AI-Rad for the detection of pleural effusions is lower compared to the WR (0.74 versus 0.88). The negative-predictive-values (NPV) of the AI-Rad for the detection of all pre-defined findings are on a high level and comparable to the WR. The seemingly advantageous high sensitivity of the AI-Rad is partially offset by the disadvantage of a high false-detection-rate. At the current stage of development, therefore, the high NPVs may be the greatest benefit of the AI-Rad giving radiologists the possibility to re-insure their own negative search for pathologies and thus boosting their confidence in their reports.https://doi.org/10.1038/s41598-023-30521-2 |
| spellingShingle | Julius Henning Niehoff Jana Kalaitzidis Jan Robert Kroeger Denise Schoenbeck Jan Borggrefe Arwed Elias Michael Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays Scientific Reports |
| title | Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays |
| title_full | Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays |
| title_fullStr | Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays |
| title_full_unstemmed | Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays |
| title_short | Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-rays |
| title_sort | evaluation of the clinical performance of an ai based application for the automated analysis of chest x rays |
| url | https://doi.org/10.1038/s41598-023-30521-2 |
| work_keys_str_mv | AT juliushenningniehoff evaluationoftheclinicalperformanceofanaibasedapplicationfortheautomatedanalysisofchestxrays AT janakalaitzidis evaluationoftheclinicalperformanceofanaibasedapplicationfortheautomatedanalysisofchestxrays AT janrobertkroeger evaluationoftheclinicalperformanceofanaibasedapplicationfortheautomatedanalysisofchestxrays AT deniseschoenbeck evaluationoftheclinicalperformanceofanaibasedapplicationfortheautomatedanalysisofchestxrays AT janborggrefe evaluationoftheclinicalperformanceofanaibasedapplicationfortheautomatedanalysisofchestxrays AT arwedeliasmichael evaluationoftheclinicalperformanceofanaibasedapplicationfortheautomatedanalysisofchestxrays |