Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review
Background: Deep learning (DL)-based models have demonstrated an ability to automatically diagnose clinically significant prostate cancer (PCa) on MRI scans and are regularly reported to approach expert performance. The aim of this work was to systematically review the literature comparing deep lear...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Life |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1729/12/10/1490 |
| _version_ | 1797472023143776256 |
|---|---|
| author | Christian Roest Stefan J Fransen Thomas C Kwee Derya Yakar |
| author_facet | Christian Roest Stefan J Fransen Thomas C Kwee Derya Yakar |
| author_sort | Christian Roest |
| collection | DOAJ |
| description | Background: Deep learning (DL)-based models have demonstrated an ability to automatically diagnose clinically significant prostate cancer (PCa) on MRI scans and are regularly reported to approach expert performance. The aim of this work was to systematically review the literature comparing deep learning (DL) systems to radiologists in order to evaluate the comparative performance of current state-of-the-art deep learning models and radiologists. Methods: This systematic review was conducted in accordance with the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Studies investigating DL models for diagnosing clinically significant (cs) PCa on MRI were included. The quality and risk of bias of each study were assessed using the checklist for AI in medical imaging (CLAIM) and QUADAS-2, respectively. Patient level and lesion-based diagnostic performance were separately evaluated by comparing the sensitivity achieved by DL and radiologists at an identical specificity and the false positives per patient, respectively. Results: The final selection consisted of eight studies with a combined 7337 patients. The median study quality with CLAIM was 74.1% (IQR: 70.6–77.6). DL achieved an identical patient-level performance to the radiologists for PI-RADS ≥ 3 (both 97.7%, SD = 2.1%). DL had a lower sensitivity for PI-RADS ≥ 4 (84.2% vs. 88.8%, <i>p</i> = 0.43). The sensitivity of DL for lesion localization was also between 2% and 12.5% lower than that of the radiologists. Conclusions: DL models for the diagnosis of csPCa on MRI appear to approach the performance of experts but currently have a lower sensitivity compared to experienced radiologists. There is a need for studies with larger datasets and for validation on external data. |
| first_indexed | 2024-03-09T19:56:34Z |
| format | Article |
| id | doaj.art-bf24054bb50045ef8bab2ca7a9be7187 |
| institution | Directory Open Access Journal |
| issn | 2075-1729 |
| language | English |
| last_indexed | 2024-03-09T19:56:34Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Life |
| spelling | doaj.art-bf24054bb50045ef8bab2ca7a9be71872023-11-24T00:55:47ZengMDPI AGLife2075-17292022-09-011210149010.3390/life12101490Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic ReviewChristian Roest0Stefan J Fransen1Thomas C Kwee2Derya Yakar3Medical Imaging Center, Departments of Radiology, Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700 RB Groningen, The NetherlandsMedical Imaging Center, Departments of Radiology, Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700 RB Groningen, The NetherlandsMedical Imaging Center, Departments of Radiology, Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700 RB Groningen, The NetherlandsMedical Imaging Center, Departments of Radiology, Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700 RB Groningen, The NetherlandsBackground: Deep learning (DL)-based models have demonstrated an ability to automatically diagnose clinically significant prostate cancer (PCa) on MRI scans and are regularly reported to approach expert performance. The aim of this work was to systematically review the literature comparing deep learning (DL) systems to radiologists in order to evaluate the comparative performance of current state-of-the-art deep learning models and radiologists. Methods: This systematic review was conducted in accordance with the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Studies investigating DL models for diagnosing clinically significant (cs) PCa on MRI were included. The quality and risk of bias of each study were assessed using the checklist for AI in medical imaging (CLAIM) and QUADAS-2, respectively. Patient level and lesion-based diagnostic performance were separately evaluated by comparing the sensitivity achieved by DL and radiologists at an identical specificity and the false positives per patient, respectively. Results: The final selection consisted of eight studies with a combined 7337 patients. The median study quality with CLAIM was 74.1% (IQR: 70.6–77.6). DL achieved an identical patient-level performance to the radiologists for PI-RADS ≥ 3 (both 97.7%, SD = 2.1%). DL had a lower sensitivity for PI-RADS ≥ 4 (84.2% vs. 88.8%, <i>p</i> = 0.43). The sensitivity of DL for lesion localization was also between 2% and 12.5% lower than that of the radiologists. Conclusions: DL models for the diagnosis of csPCa on MRI appear to approach the performance of experts but currently have a lower sensitivity compared to experienced radiologists. There is a need for studies with larger datasets and for validation on external data.https://www.mdpi.com/2075-1729/12/10/1490prostatic neoplasmsmagnetic resonance imagingdeep learning |
| spellingShingle | Christian Roest Stefan J Fransen Thomas C Kwee Derya Yakar Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review Life prostatic neoplasms magnetic resonance imaging deep learning |
| title | Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review |
| title_full | Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review |
| title_fullStr | Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review |
| title_full_unstemmed | Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review |
| title_short | Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review |
| title_sort | comparative performance of deep learning and radiologists for the diagnosis and localization of clinically significant prostate cancer at mri a systematic review |
| topic | prostatic neoplasms magnetic resonance imaging deep learning |
| url | https://www.mdpi.com/2075-1729/12/10/1490 |
| work_keys_str_mv | AT christianroest comparativeperformanceofdeeplearningandradiologistsforthediagnosisandlocalizationofclinicallysignificantprostatecanceratmriasystematicreview AT stefanjfransen comparativeperformanceofdeeplearningandradiologistsforthediagnosisandlocalizationofclinicallysignificantprostatecanceratmriasystematicreview AT thomasckwee comparativeperformanceofdeeplearningandradiologistsforthediagnosisandlocalizationofclinicallysignificantprostatecanceratmriasystematicreview AT deryayakar comparativeperformanceofdeeplearningandradiologistsforthediagnosisandlocalizationofclinicallysignificantprostatecanceratmriasystematicreview |