Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography
PurposeTo compare the mammographic malignant architectural distortion (AD) detection performance of radiologists who read mammographic examinations unaided versus those who read these examinations with the support of artificial intelligence (AI) systems.Material and MethodsThis retrospective case-co...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-04-01
|
| Series: | Frontiers in Oncology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fonc.2022.880150/full |
| _version_ | 1811330561740898304 |
|---|---|
| author | Yun Wan Yunfei Tong Yunfei Tong Yuanyuan Liu Yan Huang Guoyan Yao Daniel Q. Chen Bo Liu |
| author_facet | Yun Wan Yunfei Tong Yunfei Tong Yuanyuan Liu Yan Huang Guoyan Yao Daniel Q. Chen Bo Liu |
| author_sort | Yun Wan |
| collection | DOAJ |
| description | PurposeTo compare the mammographic malignant architectural distortion (AD) detection performance of radiologists who read mammographic examinations unaided versus those who read these examinations with the support of artificial intelligence (AI) systems.Material and MethodsThis retrospective case-control study was based on a double-reading of clinical mammograms between January 2011 and December 2016 at a large tertiary academic medical center. The study included 177 malignant and 90 benign architectural distortion (AD) patients. The model was built based on the ResNeXt-50 network. Algorithms used deep learning convolutional neural networks, feature classifiers, image analysis algorithms to depict AD and output a score that translated to malignant. The accuracy for malignant AD detection was evaluated using area under the curve (AUC).ResultsThe overall AUC was 0.733 (95% CI, 0.673-0.792) for Reader First-1, 0.652 (95% CI, 0.586-0.717) for Reader First-2, and 0.655 (95% CI, 0.590-0.719) for Reader First-3. and the overall AUCs for Reader Second-1, 2, 3 were 0.875 (95% CI, 0.830-0.919), 0.882 (95% CI, 0.839-0.926), 0.884 (95% CI, 0.841-0.927),respectively. The AUCs for all the reader-second radiologists were significantly higher than those for all the reader-first radiologists (Reader First-1 vs. Reader Second-1, P= 0.004). The overall AUC was 0.792 (95% CI, 0.660-0.925) for AI algorithms. The combination assessment of AI algorithms and Reader First-1 achieved an AUC of 0.880 (95% CI, 0.793-0.968), increased than the Reader First-1 alone and AI algorithms alone. AI algorithms alone achieved a specificity of 61.1% and a sensitivity of 80.6%. The specificity for Reader First-1 was 55.5%, and the sensitivity was 86.1%. The results of the combined assessment of AI and Reader First-1 showed a specificity of 72.7% and sensitivity of 91.7%. The performance showed significant improvements compared with AI alone (p<0.001) as well as the reader first-1 alone (p=0.006).ConclusionWhile the single AI algorithm did not outperform radiologists, an ensemble of AI algorithms combined with junior radiologist assessments were found to improve the overall accuracy. This study underscores the potential of using machine learning methods to enhance mammography interpretation, especially in remote areas and primary hospitals. |
| first_indexed | 2024-04-13T16:05:14Z |
| format | Article |
| id | doaj.art-119791fbd2c54f6ba6986469216def7f |
| institution | Directory Open Access Journal |
| issn | 2234-943X |
| language | English |
| last_indexed | 2024-04-13T16:05:14Z |
| publishDate | 2022-04-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Oncology |
| spelling | doaj.art-119791fbd2c54f6ba6986469216def7f2022-12-22T02:40:25ZengFrontiers Media S.A.Frontiers in Oncology2234-943X2022-04-011210.3389/fonc.2022.880150880150Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on MammographyYun Wan0Yunfei Tong1Yunfei Tong2Yuanyuan Liu3Yan Huang4Guoyan Yao5Daniel Q. Chen6Bo Liu7Department of Radiology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, ChinaAI Research Lab, Boston Meditech Group, Burlington, MA, United StatesAI Research Lab, Shanghai Yanghe Huajian Artificial Intelligence Technology Co., Ltd, Shanghai, ChinaDepartment of Radiology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, ChinaDepartment of Radiology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, ChinaDepartment of Radiology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, ChinaAI Research Lab, Boston Meditech Group, Burlington, MA, United StatesDepartment of Radiology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, ChinaPurposeTo compare the mammographic malignant architectural distortion (AD) detection performance of radiologists who read mammographic examinations unaided versus those who read these examinations with the support of artificial intelligence (AI) systems.Material and MethodsThis retrospective case-control study was based on a double-reading of clinical mammograms between January 2011 and December 2016 at a large tertiary academic medical center. The study included 177 malignant and 90 benign architectural distortion (AD) patients. The model was built based on the ResNeXt-50 network. Algorithms used deep learning convolutional neural networks, feature classifiers, image analysis algorithms to depict AD and output a score that translated to malignant. The accuracy for malignant AD detection was evaluated using area under the curve (AUC).ResultsThe overall AUC was 0.733 (95% CI, 0.673-0.792) for Reader First-1, 0.652 (95% CI, 0.586-0.717) for Reader First-2, and 0.655 (95% CI, 0.590-0.719) for Reader First-3. and the overall AUCs for Reader Second-1, 2, 3 were 0.875 (95% CI, 0.830-0.919), 0.882 (95% CI, 0.839-0.926), 0.884 (95% CI, 0.841-0.927),respectively. The AUCs for all the reader-second radiologists were significantly higher than those for all the reader-first radiologists (Reader First-1 vs. Reader Second-1, P= 0.004). The overall AUC was 0.792 (95% CI, 0.660-0.925) for AI algorithms. The combination assessment of AI algorithms and Reader First-1 achieved an AUC of 0.880 (95% CI, 0.793-0.968), increased than the Reader First-1 alone and AI algorithms alone. AI algorithms alone achieved a specificity of 61.1% and a sensitivity of 80.6%. The specificity for Reader First-1 was 55.5%, and the sensitivity was 86.1%. The results of the combined assessment of AI and Reader First-1 showed a specificity of 72.7% and sensitivity of 91.7%. The performance showed significant improvements compared with AI alone (p<0.001) as well as the reader first-1 alone (p=0.006).ConclusionWhile the single AI algorithm did not outperform radiologists, an ensemble of AI algorithms combined with junior radiologist assessments were found to improve the overall accuracy. This study underscores the potential of using machine learning methods to enhance mammography interpretation, especially in remote areas and primary hospitals.https://www.frontiersin.org/articles/10.3389/fonc.2022.880150/fullartificial intelligencearchitectural distortionmammographybreast cancermalignant |
| spellingShingle | Yun Wan Yunfei Tong Yunfei Tong Yuanyuan Liu Yan Huang Guoyan Yao Daniel Q. Chen Bo Liu Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography Frontiers in Oncology artificial intelligence architectural distortion mammography breast cancer malignant |
| title | Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography |
| title_full | Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography |
| title_fullStr | Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography |
| title_full_unstemmed | Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography |
| title_short | Evaluation of the Combination of Artificial Intelligence and Radiologist Assessments to Interpret Malignant Architectural Distortion on Mammography |
| title_sort | evaluation of the combination of artificial intelligence and radiologist assessments to interpret malignant architectural distortion on mammography |
| topic | artificial intelligence architectural distortion mammography breast cancer malignant |
| url | https://www.frontiersin.org/articles/10.3389/fonc.2022.880150/full |
| work_keys_str_mv | AT yunwan evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT yunfeitong evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT yunfeitong evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT yuanyuanliu evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT yanhuang evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT guoyanyao evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT danielqchen evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography AT boliu evaluationofthecombinationofartificialintelligenceandradiologistassessmentstointerpretmalignantarchitecturaldistortiononmammography |