Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context
Summary: Background: For patients with early-stage breast cancers, neoadjuvant treatment is recommended for non-luminal tumors instead of luminal tumors. Preoperative distinguish between luminal and non-luminal cancers at early stages will facilitate treatment decisions making. However, the molecul...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-08-01
|
| Series: | EBioMedicine |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352396423002712 |
| _version_ | 1797745988743462912 |
|---|---|
| author | Yini Huang Zhao Yao Lingling Li Rushuang Mao Weijun Huang Zhengming Hu Yixin Hu Yun Wang Ruohan Guo Xiaofeng Tang Liang Yang Yuanyuan Wang Rongzhen Luo Jinhua Yu Jianhua Zhou |
| author_facet | Yini Huang Zhao Yao Lingling Li Rushuang Mao Weijun Huang Zhengming Hu Yixin Hu Yun Wang Ruohan Guo Xiaofeng Tang Liang Yang Yuanyuan Wang Rongzhen Luo Jinhua Yu Jianhua Zhou |
| author_sort | Yini Huang |
| collection | DOAJ |
| description | Summary: Background: For patients with early-stage breast cancers, neoadjuvant treatment is recommended for non-luminal tumors instead of luminal tumors. Preoperative distinguish between luminal and non-luminal cancers at early stages will facilitate treatment decisions making. However, the molecular immunohistochemical subtypes based on biopsy specimens are not always consistent with final results based on surgical specimens due to the high intra-tumoral heterogeneity. Given that, we aimed to develop and validate a deep learning radiopathomics (DLRP) model to preoperatively distinguish between luminal and non-luminal breast cancers at early stages based on preoperative ultrasound (US) images, and hematoxylin and eosin (H&E)-stained biopsy slides. Methods: This multicentre study included three cohorts from a prospective study conducted by our team and registered on the Chinese Clinical Trial Registry (ChiCTR1900027497). Between January 2019 and August 2021, 1809 US images and 603 H&E-stained whole slide images (WSIs) from 603 patients with early-stage breast cancers were obtained. A Resnet18 model pre-trained on ImageNet and a multi-instance learning based attention model were used to extract the features of US and WSIs, respectively. An US-guided Co-Attention module (UCA) was designed for feature fusion of US and WSIs. The DLRP model was constructed based on these three feature sets including deep learning US feature, deep learning WSIs feature and UCA-fused feature from a training cohort (1467 US images and 489 WSIs from 489 patients). The DLRP model's diagnostic performance was validated in an internal validation cohort (342 US images and 114 WSIs from 114 patients) and an external test cohort (270 US images and 90 WSIs from 90 patients). We also compared diagnostic efficacy of the DLRP model with that of deep learning radiomics model and deep learning pathomics model in the external test cohort. Findings: The DLRP yielded high performance with area under the curve (AUC) values of 0.929 (95% CI 0.865–0.968) in the internal validation cohort, and 0.900 (95% CI 0.819–0.953) in the external test cohort. The DLRP also outperformed deep learning radiomics model based on US images only (AUC 0.815 [0.719–0.889], p = 0.027) and deep learning pathomics model based on WSIs only (AUC 0.802 [0.704–0.878], p = 0.013) in the external test cohort. Interpretation: The DLRP can effectively distinguish between luminal and non-luminal breast cancers at early stages before surgery based on pretherapeutic US images and biopsy H&E-stained WSIs, providing a tool to facilitate treatment decision making in early-stage breast cancers. Funding: Natural Science Foundation of Guangdong Province (No. 2023A1515011564), and National Natural Science Foundation of China (No. 91959127; No. 81971631). |
| first_indexed | 2024-03-12T15:30:41Z |
| format | Article |
| id | doaj.art-96a09d1663fa421c9a703b588e9174af |
| institution | Directory Open Access Journal |
| issn | 2352-3964 |
| language | English |
| last_indexed | 2024-03-12T15:30:41Z |
| publishDate | 2023-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | EBioMedicine |
| spelling | doaj.art-96a09d1663fa421c9a703b588e9174af2023-08-10T04:34:31ZengElsevierEBioMedicine2352-39642023-08-0194104706Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in contextYini Huang0Zhao Yao1Lingling Li2Rushuang Mao3Weijun Huang4Zhengming Hu5Yixin Hu6Yun Wang7Ruohan Guo8Xiaofeng Tang9Liang Yang10Yuanyuan Wang11Rongzhen Luo12Jinhua Yu13Jianhua Zhou14Department of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaSchool of Information Science and Technology, Fudan University, Shanghai, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaDivision of Interventional Ultrasound, Department of Medical Ultrasonics, Foshan First People's Hospital, Foshan, Guangdong, ChinaDepartment of Ultrasound, Peking University Shenzhen Hospital, Shenzhen, Guangdong, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaDepartment of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, ChinaSchool of Information Science and Technology, Fudan University, Shanghai, ChinaDepartment of Pathology, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China; Corresponding author. Department of Pathology, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, No. 651 Dongfeng Road East, Guangzhou, China.School of Information Science and Technology, Fudan University, Shanghai, China; Corresponding author. School of Information Science and Technology, Fudan University, 220 Handan Road, 200433, Shanghai, China.Department of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China; Corresponding author. Department of Ultrasound, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, No. 651 Dongfeng Road East, Guangzhou, China.Summary: Background: For patients with early-stage breast cancers, neoadjuvant treatment is recommended for non-luminal tumors instead of luminal tumors. Preoperative distinguish between luminal and non-luminal cancers at early stages will facilitate treatment decisions making. However, the molecular immunohistochemical subtypes based on biopsy specimens are not always consistent with final results based on surgical specimens due to the high intra-tumoral heterogeneity. Given that, we aimed to develop and validate a deep learning radiopathomics (DLRP) model to preoperatively distinguish between luminal and non-luminal breast cancers at early stages based on preoperative ultrasound (US) images, and hematoxylin and eosin (H&E)-stained biopsy slides. Methods: This multicentre study included three cohorts from a prospective study conducted by our team and registered on the Chinese Clinical Trial Registry (ChiCTR1900027497). Between January 2019 and August 2021, 1809 US images and 603 H&E-stained whole slide images (WSIs) from 603 patients with early-stage breast cancers were obtained. A Resnet18 model pre-trained on ImageNet and a multi-instance learning based attention model were used to extract the features of US and WSIs, respectively. An US-guided Co-Attention module (UCA) was designed for feature fusion of US and WSIs. The DLRP model was constructed based on these three feature sets including deep learning US feature, deep learning WSIs feature and UCA-fused feature from a training cohort (1467 US images and 489 WSIs from 489 patients). The DLRP model's diagnostic performance was validated in an internal validation cohort (342 US images and 114 WSIs from 114 patients) and an external test cohort (270 US images and 90 WSIs from 90 patients). We also compared diagnostic efficacy of the DLRP model with that of deep learning radiomics model and deep learning pathomics model in the external test cohort. Findings: The DLRP yielded high performance with area under the curve (AUC) values of 0.929 (95% CI 0.865–0.968) in the internal validation cohort, and 0.900 (95% CI 0.819–0.953) in the external test cohort. The DLRP also outperformed deep learning radiomics model based on US images only (AUC 0.815 [0.719–0.889], p = 0.027) and deep learning pathomics model based on WSIs only (AUC 0.802 [0.704–0.878], p = 0.013) in the external test cohort. Interpretation: The DLRP can effectively distinguish between luminal and non-luminal breast cancers at early stages before surgery based on pretherapeutic US images and biopsy H&E-stained WSIs, providing a tool to facilitate treatment decision making in early-stage breast cancers. Funding: Natural Science Foundation of Guangdong Province (No. 2023A1515011564), and National Natural Science Foundation of China (No. 91959127; No. 81971631).http://www.sciencedirect.com/science/article/pii/S2352396423002712Breast cancerUltrasoundWhole slide imagingDeep learning |
| spellingShingle | Yini Huang Zhao Yao Lingling Li Rushuang Mao Weijun Huang Zhengming Hu Yixin Hu Yun Wang Ruohan Guo Xiaofeng Tang Liang Yang Yuanyuan Wang Rongzhen Luo Jinhua Yu Jianhua Zhou Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context EBioMedicine Breast cancer Ultrasound Whole slide imaging Deep learning |
| title | Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context |
| title_full | Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context |
| title_fullStr | Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context |
| title_full_unstemmed | Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context |
| title_short | Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancersResearch in context |
| title_sort | deep learning radiopathomics based on preoperative us images and biopsy whole slide images can distinguish between luminal and non luminal tumors in early stage breast cancersresearch in context |
| topic | Breast cancer Ultrasound Whole slide imaging Deep learning |
| url | http://www.sciencedirect.com/science/article/pii/S2352396423002712 |
| work_keys_str_mv | AT yinihuang deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT zhaoyao deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT linglingli deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT rushuangmao deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT weijunhuang deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT zhengminghu deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT yixinhu deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT yunwang deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT ruohanguo deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT xiaofengtang deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT liangyang deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT yuanyuanwang deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT rongzhenluo deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT jinhuayu deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext AT jianhuazhou deeplearningradiopathomicsbasedonpreoperativeusimagesandbiopsywholeslideimagescandistinguishbetweenluminalandnonluminaltumorsinearlystagebreastcancersresearchincontext |