Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists
It is unclear whether the visualization methods for artificial-intelligence-based computer-aided detection (AI-CAD) of chest radiographs influence the accuracy of readers’ interpretation. We aimed to evaluate the accuracy of radiologists’ interpretations of chest radiographs using different visualiz...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Diagnostics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4418/13/6/1089 |
| _version_ | 1827750286111801344 |
|---|---|
| author | Sungho Hong Eui Jin Hwang Soojin Kim Jiyoung Song Taehee Lee Gyeong Deok Jo Yelim Choi Chang Min Park Jin Mo Goo |
| author_facet | Sungho Hong Eui Jin Hwang Soojin Kim Jiyoung Song Taehee Lee Gyeong Deok Jo Yelim Choi Chang Min Park Jin Mo Goo |
| author_sort | Sungho Hong |
| collection | DOAJ |
| description | It is unclear whether the visualization methods for artificial-intelligence-based computer-aided detection (AI-CAD) of chest radiographs influence the accuracy of readers’ interpretation. We aimed to evaluate the accuracy of radiologists’ interpretations of chest radiographs using different visualization methods for the same AI-CAD. Initial chest radiographs of patients with acute respiratory symptoms were retrospectively collected. A commercialized AI-CAD using three different methods of visualizing was applied: (a) closed-line method, (b) heat map method, and (c) combined method. A reader test was conducted with five trainee radiologists over three interpretation sessions. In each session, the chest radiographs were interpreted using AI-CAD with one of the three visualization methods in random order. Examination-level sensitivity and accuracy, and lesion-level detection rates for clinically significant abnormalities were evaluated for the three visualization methods. The sensitivity (<i>p</i> = 0.007) and accuracy (<i>p</i> = 0.037) of the combined method are significantly higher than that of the closed-line method. Detection rates using the heat map method (<i>p</i> = 0.043) and the combined method (<i>p</i> = 0.004) are significantly higher than those using the closed-line method. The methods for visualizing AI-CAD results for chest radiographs influenced the performance of radiologists’ interpretations. Combining the closed-line and heat map methods for visualizing AI-CAD results led to the highest sensitivity and accuracy of radiologists. |
| first_indexed | 2024-03-11T06:41:28Z |
| format | Article |
| id | doaj.art-10f825a99b304adfa3fa412a470b59eb |
| institution | Directory Open Access Journal |
| issn | 2075-4418 |
| language | English |
| last_indexed | 2024-03-11T06:41:28Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Diagnostics |
| spelling | doaj.art-10f825a99b304adfa3fa412a470b59eb2023-11-17T10:34:11ZengMDPI AGDiagnostics2075-44182023-03-01136108910.3390/diagnostics13061089Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of RadiologistsSungho Hong0Eui Jin Hwang1Soojin Kim2Jiyoung Song3Taehee Lee4Gyeong Deok Jo5Yelim Choi6Chang Min Park7Jin Mo Goo8Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaDepartment of Radiology, Seoul National University Hospital, Seoul 03082, Republic of KoreaIt is unclear whether the visualization methods for artificial-intelligence-based computer-aided detection (AI-CAD) of chest radiographs influence the accuracy of readers’ interpretation. We aimed to evaluate the accuracy of radiologists’ interpretations of chest radiographs using different visualization methods for the same AI-CAD. Initial chest radiographs of patients with acute respiratory symptoms were retrospectively collected. A commercialized AI-CAD using three different methods of visualizing was applied: (a) closed-line method, (b) heat map method, and (c) combined method. A reader test was conducted with five trainee radiologists over three interpretation sessions. In each session, the chest radiographs were interpreted using AI-CAD with one of the three visualization methods in random order. Examination-level sensitivity and accuracy, and lesion-level detection rates for clinically significant abnormalities were evaluated for the three visualization methods. The sensitivity (<i>p</i> = 0.007) and accuracy (<i>p</i> = 0.037) of the combined method are significantly higher than that of the closed-line method. Detection rates using the heat map method (<i>p</i> = 0.043) and the combined method (<i>p</i> = 0.004) are significantly higher than those using the closed-line method. The methods for visualizing AI-CAD results for chest radiographs influenced the performance of radiologists’ interpretations. Combining the closed-line and heat map methods for visualizing AI-CAD results led to the highest sensitivity and accuracy of radiologists.https://www.mdpi.com/2075-4418/13/6/1089chest radiographyartificial intelligencedeep learningcomputer-aided detectiondiagnostic accuracy |
| spellingShingle | Sungho Hong Eui Jin Hwang Soojin Kim Jiyoung Song Taehee Lee Gyeong Deok Jo Yelim Choi Chang Min Park Jin Mo Goo Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists Diagnostics chest radiography artificial intelligence deep learning computer-aided detection diagnostic accuracy |
| title | Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists |
| title_full | Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists |
| title_fullStr | Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists |
| title_full_unstemmed | Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists |
| title_short | Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists |
| title_sort | methods of visualizing the results of an artificial intelligence based computer aided detection system for chest radiographs effect on the diagnostic performance of radiologists |
| topic | chest radiography artificial intelligence deep learning computer-aided detection diagnostic accuracy |
| url | https://www.mdpi.com/2075-4418/13/6/1089 |
| work_keys_str_mv | AT sunghohong methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT euijinhwang methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT soojinkim methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT jiyoungsong methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT taeheelee methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT gyeongdeokjo methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT yelimchoi methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT changminpark methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists AT jinmogoo methodsofvisualizingtheresultsofanartificialintelligencebasedcomputeraideddetectionsystemforchestradiographseffectonthediagnosticperformanceofradiologists |