Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital
Purpose: The aim of our study is to evaluate artificial intelligence (AI) support in pelvic fracture diagnosis on X-rays, focusing on performance, workflow integration and radiologists’ feedback in a spoke emergency hospital. Materials and methods: Between August and November 2021, a total of 235 si...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-12-01
|
| Series: | European Journal of Radiology Open |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352047723000308 |
| _version_ | 1827591630021984256 |
|---|---|
| author | Francesca Rosa Duccio Buccicardi Adolfo Romano Fabio Borda Maria Chiara D’Auria Alessandro Gastaldo |
| author_facet | Francesca Rosa Duccio Buccicardi Adolfo Romano Fabio Borda Maria Chiara D’Auria Alessandro Gastaldo |
| author_sort | Francesca Rosa |
| collection | DOAJ |
| description | Purpose: The aim of our study is to evaluate artificial intelligence (AI) support in pelvic fracture diagnosis on X-rays, focusing on performance, workflow integration and radiologists’ feedback in a spoke emergency hospital. Materials and methods: Between August and November 2021, a total of 235 sites of fracture or suspected fracture were evaluated and enrolled in the prospective study. Radiologist’s specificity, sensibility accuracy, positive and negative predictive values were compared to AI. Cohen's kappa was used to calculate the agreement between AI and radiologist. We also reviewed the AI workflow integration process, focusing on potential issues and assessed radiologists’ opinion on AI via a survey. Results: The radiologist performance in accuracy, sensitivity and specificity was better than AI but McNemar test demonstrated no statistically significant difference between AI and radiologist’s performance (p = 0.32). Calculated Cohen’s K of 0.64. Conclusion: Contrary to expectations, our preliminary results did not prove a real improvement of patient outcome nor in reporting time but demonstrated AI high NPV (94,62%) and non-inferiority to radiologist performance. Moreover, the commercially available AI algorithm used in our study automatically learn from data and so we expect a progressive performance improvement. AI could be considered as a promising tool to rule-out fractures (especially when used as a “second reader”) and to prioritize positive cases, especially in increasing workload scenarios (ED, nightshifts) but further research is needed to evaluate the real impact on the clinical practice. |
| first_indexed | 2024-03-09T01:34:28Z |
| format | Article |
| id | doaj.art-ab78ba2bcc264fff94b7d69f75777189 |
| institution | Directory Open Access Journal |
| issn | 2352-0477 |
| language | English |
| last_indexed | 2024-03-09T01:34:28Z |
| publishDate | 2023-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | European Journal of Radiology Open |
| spelling | doaj.art-ab78ba2bcc264fff94b7d69f757771892023-12-09T06:06:36ZengElsevierEuropean Journal of Radiology Open2352-04772023-12-0111100504Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospitalFrancesca Rosa0Duccio Buccicardi1Adolfo Romano2Fabio Borda3Maria Chiara D’Auria4Alessandro Gastaldo5Diagnostic Imaging Department, San Paolo Hospital, ASL 2, via Genova 30, Savona, Italy; Italian Society of Medical and Interventional Radiology (SIRM), SIRM Foundation, Milan, Italy; Corresponding author at: Diagnostic Imaging Department, San Paolo Hospital, ASL 2, via Genova 30, Savona, Italy.Diagnostic Imaging Department, San Paolo Hospital, ASL 2, via Genova 30, Savona, Italy; Italian Society of Medical and Interventional Radiology (SIRM), SIRM Foundation, Milan, ItalyDiagnostic Imaging Department, San Paolo Hospital, ASL 2, via Genova 30, Savona, ItalyDepartment of Health Sciences (DISSAL) – Radiology Section, University of Genoa, 16132 Genoa, ItalyDiagnostic Imaging Department, San Paolo Hospital, ASL 2, via Genova 30, Savona, ItalyDiagnostic Imaging Department, San Paolo Hospital, ASL 2, via Genova 30, Savona, ItalyPurpose: The aim of our study is to evaluate artificial intelligence (AI) support in pelvic fracture diagnosis on X-rays, focusing on performance, workflow integration and radiologists’ feedback in a spoke emergency hospital. Materials and methods: Between August and November 2021, a total of 235 sites of fracture or suspected fracture were evaluated and enrolled in the prospective study. Radiologist’s specificity, sensibility accuracy, positive and negative predictive values were compared to AI. Cohen's kappa was used to calculate the agreement between AI and radiologist. We also reviewed the AI workflow integration process, focusing on potential issues and assessed radiologists’ opinion on AI via a survey. Results: The radiologist performance in accuracy, sensitivity and specificity was better than AI but McNemar test demonstrated no statistically significant difference between AI and radiologist’s performance (p = 0.32). Calculated Cohen’s K of 0.64. Conclusion: Contrary to expectations, our preliminary results did not prove a real improvement of patient outcome nor in reporting time but demonstrated AI high NPV (94,62%) and non-inferiority to radiologist performance. Moreover, the commercially available AI algorithm used in our study automatically learn from data and so we expect a progressive performance improvement. AI could be considered as a promising tool to rule-out fractures (especially when used as a “second reader”) and to prioritize positive cases, especially in increasing workload scenarios (ED, nightshifts) but further research is needed to evaluate the real impact on the clinical practice.http://www.sciencedirect.com/science/article/pii/S2352047723000308Artificial intelligencePlain radiographFractureComputed tomography |
| spellingShingle | Francesca Rosa Duccio Buccicardi Adolfo Romano Fabio Borda Maria Chiara D’Auria Alessandro Gastaldo Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital European Journal of Radiology Open Artificial intelligence Plain radiograph Fracture Computed tomography |
| title | Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital |
| title_full | Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital |
| title_fullStr | Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital |
| title_full_unstemmed | Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital |
| title_short | Artificial intelligence and pelvic fracture diagnosis on X-rays: a preliminary study on performance, workflow integration and radiologists' feedback assessment in a spoke emergency hospital |
| title_sort | artificial intelligence and pelvic fracture diagnosis on x rays a preliminary study on performance workflow integration and radiologists feedback assessment in a spoke emergency hospital |
| topic | Artificial intelligence Plain radiograph Fracture Computed tomography |
| url | http://www.sciencedirect.com/science/article/pii/S2352047723000308 |
| work_keys_str_mv | AT francescarosa artificialintelligenceandpelvicfracturediagnosisonxraysapreliminarystudyonperformanceworkflowintegrationandradiologistsfeedbackassessmentinaspokeemergencyhospital AT ducciobuccicardi artificialintelligenceandpelvicfracturediagnosisonxraysapreliminarystudyonperformanceworkflowintegrationandradiologistsfeedbackassessmentinaspokeemergencyhospital AT adolforomano artificialintelligenceandpelvicfracturediagnosisonxraysapreliminarystudyonperformanceworkflowintegrationandradiologistsfeedbackassessmentinaspokeemergencyhospital AT fabioborda artificialintelligenceandpelvicfracturediagnosisonxraysapreliminarystudyonperformanceworkflowintegrationandradiologistsfeedbackassessmentinaspokeemergencyhospital AT mariachiaradauria artificialintelligenceandpelvicfracturediagnosisonxraysapreliminarystudyonperformanceworkflowintegrationandradiologistsfeedbackassessmentinaspokeemergencyhospital AT alessandrogastaldo artificialintelligenceandpelvicfracturediagnosisonxraysapreliminarystudyonperformanceworkflowintegrationandradiologistsfeedbackassessmentinaspokeemergencyhospital |