Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging
We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patien...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Bioengineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2306-5354/10/12/1340 |
| _version_ | 1827575608408801280 |
|---|---|
| author | Liang Jin Zhuangxuan Ma Haiqing Li Feng Gao Pan Gao Nan Yang Dechun Li Ming Li Daoying Geng |
| author_facet | Liang Jin Zhuangxuan Ma Haiqing Li Feng Gao Pan Gao Nan Yang Dechun Li Ming Li Daoying Geng |
| author_sort | Liang Jin |
| collection | DOAJ |
| description | We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at <i>p</i> < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (<i>p</i> < 0.05). Automatic segmentation was more consistent than manual segmentation (<i>p</i> < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features. |
| first_indexed | 2024-03-08T20:59:28Z |
| format | Article |
| id | doaj.art-34e6b6c93c9145119747a9ffb2aa9e86 |
| institution | Directory Open Access Journal |
| issn | 2306-5354 |
| language | English |
| last_indexed | 2024-03-08T20:59:28Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Bioengineering |
| spelling | doaj.art-34e6b6c93c9145119747a9ffb2aa9e862023-12-22T13:53:56ZengMDPI AGBioengineering2306-53542023-11-011012134010.3390/bioengineering10121340Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance ImagingLiang Jin0Zhuangxuan Ma1Haiqing Li2Feng Gao3Pan Gao4Nan Yang5Dechun Li6Ming Li7Daoying Geng8Radiology Department, Huashan Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huashan Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaRadiology Department, Huashan Hospital, Affiliated with Fudan University, Shanghai 200040, ChinaWe aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at <i>p</i> < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (<i>p</i> < 0.05). Automatic segmentation was more consistent than manual segmentation (<i>p</i> < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features.https://www.mdpi.com/2306-5354/10/12/1340prostateradiomicsinterobserver agreementautomatic segmentationT2-weighted imaging |
| spellingShingle | Liang Jin Zhuangxuan Ma Haiqing Li Feng Gao Pan Gao Nan Yang Dechun Li Ming Li Daoying Geng Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging Bioengineering prostate radiomics interobserver agreement automatic segmentation T2-weighted imaging |
| title | Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging |
| title_full | Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging |
| title_fullStr | Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging |
| title_full_unstemmed | Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging |
| title_short | Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging |
| title_sort | interobserver agreement in automatic segmentation annotation of prostate magnetic resonance imaging |
| topic | prostate radiomics interobserver agreement automatic segmentation T2-weighted imaging |
| url | https://www.mdpi.com/2306-5354/10/12/1340 |
| work_keys_str_mv | AT liangjin interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT zhuangxuanma interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT haiqingli interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT fenggao interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT pangao interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT nanyang interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT dechunli interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT mingli interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging AT daoyinggeng interobserveragreementinautomaticsegmentationannotationofprostatemagneticresonanceimaging |