Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists
Abstract Background There is increasing research in using segmentation of prostate cancer to create a digital 3D model from magnetic resonance imaging (MRI) scans for purposes of education or surgical planning. However, the variation in segmentation of prostate cancer among users and potential inacc...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2020-10-01
|
| Series: | Cancer Medicine |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/cam4.3386 |
| _version_ | 1797839416513789952 |
|---|---|
| author | Michael Y. Chen Maria A. Woodruff Prokar Dasgupta Nicholas J. Rukin |
| author_facet | Michael Y. Chen Maria A. Woodruff Prokar Dasgupta Nicholas J. Rukin |
| author_sort | Michael Y. Chen |
| collection | DOAJ |
| description | Abstract Background There is increasing research in using segmentation of prostate cancer to create a digital 3D model from magnetic resonance imaging (MRI) scans for purposes of education or surgical planning. However, the variation in segmentation of prostate cancer among users and potential inaccuracy has not been studied. Methods Four consultant radiologists, four consultant urologists, four urology trainees, and four nonclinician segmentation scientists were asked to segment a single slice of a lateral T3 prostate tumor on MRI (“Prostate 1”), an anterior zone prostate tumor MRI (“Prostate 2”), and a kidney tumor computed tomography (CT) scan (“Kidney”). Time taken and self‐rated subjective accuracy out of a maximum score of 10 were recorded. Root mean square error, Dice coefficient, Matthews correlation coefficient, Jaccard index, specificity, and sensitivity were calculated using the radiologists as the ground truth. Results There was high variance among the radiologists in segmentation of Prostate 1 and 2 tumors with mean Dice coefficients of 0.81 and 0.58, respectively, compared to 0.96 for the kidney tumor. Urologists and urology trainees had similar accuracy, while nonclinicians had the lowest accuracy scores for Prostate 1 and 2 tumors (0.60 and 0.47) but similar for kidney tumor (0.95). Mean sensitivity in Prostate 1 (0.63) and Prostate 2 (0.61) was lower than specificity (0.92 and 0.93) suggesting under‐segmentation of tumors in the non‐radiologist groups. Participants spent less time on the kidney tumor segmentation and self‐rated accuracy was higher than both prostate tumors. Conclusion Segmentation of prostate cancers is more difficult than other anatomy such as kidney tumors. Less experienced participants appear to under‐segment models and underestimate the size of prostate tumors. Segmentation of prostate cancer is highly variable even among radiologists, and 3D modeling for clinical use must be performed with caution. Further work to develop a methodology to maximize segmentation accuracy is needed. |
| first_indexed | 2024-04-09T15:57:37Z |
| format | Article |
| id | doaj.art-c06843f7c806496d9bcc3e73ec45d957 |
| institution | Directory Open Access Journal |
| issn | 2045-7634 |
| language | English |
| last_indexed | 2024-04-09T15:57:37Z |
| publishDate | 2020-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | Cancer Medicine |
| spelling | doaj.art-c06843f7c806496d9bcc3e73ec45d9572023-04-25T14:00:49ZengWileyCancer Medicine2045-76342020-10-019197172718210.1002/cam4.3386Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientistsMichael Y. Chen0Maria A. Woodruff1Prokar Dasgupta2Nicholas J. Rukin3Science and Engineering Faculty Queensland University of Technology Brisbane Queensland AustraliaScience and Engineering Faculty Queensland University of Technology Brisbane Queensland AustraliaKing’s College LondonGuy’s Hospital London United KingdomScience and Engineering Faculty Queensland University of Technology Brisbane Queensland AustraliaAbstract Background There is increasing research in using segmentation of prostate cancer to create a digital 3D model from magnetic resonance imaging (MRI) scans for purposes of education or surgical planning. However, the variation in segmentation of prostate cancer among users and potential inaccuracy has not been studied. Methods Four consultant radiologists, four consultant urologists, four urology trainees, and four nonclinician segmentation scientists were asked to segment a single slice of a lateral T3 prostate tumor on MRI (“Prostate 1”), an anterior zone prostate tumor MRI (“Prostate 2”), and a kidney tumor computed tomography (CT) scan (“Kidney”). Time taken and self‐rated subjective accuracy out of a maximum score of 10 were recorded. Root mean square error, Dice coefficient, Matthews correlation coefficient, Jaccard index, specificity, and sensitivity were calculated using the radiologists as the ground truth. Results There was high variance among the radiologists in segmentation of Prostate 1 and 2 tumors with mean Dice coefficients of 0.81 and 0.58, respectively, compared to 0.96 for the kidney tumor. Urologists and urology trainees had similar accuracy, while nonclinicians had the lowest accuracy scores for Prostate 1 and 2 tumors (0.60 and 0.47) but similar for kidney tumor (0.95). Mean sensitivity in Prostate 1 (0.63) and Prostate 2 (0.61) was lower than specificity (0.92 and 0.93) suggesting under‐segmentation of tumors in the non‐radiologist groups. Participants spent less time on the kidney tumor segmentation and self‐rated accuracy was higher than both prostate tumors. Conclusion Segmentation of prostate cancers is more difficult than other anatomy such as kidney tumors. Less experienced participants appear to under‐segment models and underestimate the size of prostate tumors. Segmentation of prostate cancer is highly variable even among radiologists, and 3D modeling for clinical use must be performed with caution. Further work to develop a methodology to maximize segmentation accuracy is needed.https://doi.org/10.1002/cam4.33863D printing3D modelMRIprostatesegmentation |
| spellingShingle | Michael Y. Chen Maria A. Woodruff Prokar Dasgupta Nicholas J. Rukin Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists Cancer Medicine 3D printing 3D model MRI prostate segmentation |
| title | Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists |
| title_full | Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists |
| title_fullStr | Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists |
| title_full_unstemmed | Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists |
| title_short | Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists |
| title_sort | variability in accuracy of prostate cancer segmentation among radiologists urologists and scientists |
| topic | 3D printing 3D model MRI prostate segmentation |
| url | https://doi.org/10.1002/cam4.3386 |
| work_keys_str_mv | AT michaelychen variabilityinaccuracyofprostatecancersegmentationamongradiologistsurologistsandscientists AT mariaawoodruff variabilityinaccuracyofprostatecancersegmentationamongradiologistsurologistsandscientists AT prokardasgupta variabilityinaccuracyofprostatecancersegmentationamongradiologistsurologistsandscientists AT nicholasjrukin variabilityinaccuracyofprostatecancersegmentationamongradiologistsurologistsandscientists |