Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors
Summary: Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists’ annotation of region of interests or selection of image patches o...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-01-01
|
| Series: | iScience |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2589004222021459 |
| _version_ | 1797946181470388224 |
|---|---|
| author | Zhongxiao Li Yuwei Cong Xin Chen Jiping Qi Jingxian Sun Tao Yan He Yang Junsi Liu Enzhou Lu Lixiang Wang Jiafeng Li Hong Hu Cheng Zhang Quan Yang Jiawei Yao Penglei Yao Qiuyi Jiang Wenwu Liu Jiangning Song Lawrence Carin Yupeng Chen Shiguang Zhao Xin Gao |
| author_facet | Zhongxiao Li Yuwei Cong Xin Chen Jiping Qi Jingxian Sun Tao Yan He Yang Junsi Liu Enzhou Lu Lixiang Wang Jiafeng Li Hong Hu Cheng Zhang Quan Yang Jiawei Yao Penglei Yao Qiuyi Jiang Wenwu Liu Jiangning Song Lawrence Carin Yupeng Chen Shiguang Zhao Xin Gao |
| author_sort | Zhongxiao Li |
| collection | DOAJ |
| description | Summary: Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists’ annotation of region of interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) – based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively. |
| first_indexed | 2024-04-10T21:06:49Z |
| format | Article |
| id | doaj.art-b29bcda62de84d608bd7c769491d33c9 |
| institution | Directory Open Access Journal |
| issn | 2589-0042 |
| language | English |
| last_indexed | 2024-04-10T21:06:49Z |
| publishDate | 2023-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | iScience |
| spelling | doaj.art-b29bcda62de84d608bd7c769491d33c92023-01-22T04:41:35ZengElsevieriScience2589-00422023-01-01261105872Vision transformer-based weakly supervised histopathological image analysis of primary brain tumorsZhongxiao Li0Yuwei Cong1Xin Chen2Jiping Qi3Jingxian Sun4Tao Yan5He Yang6Junsi Liu7Enzhou Lu8Lixiang Wang9Jiafeng Li10Hong Hu11Cheng Zhang12Quan Yang13Jiawei Yao14Penglei Yao15Qiuyi Jiang16Wenwu Liu17Jiangning Song18Lawrence Carin19Yupeng Chen20Shiguang Zhao21Xin Gao22Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia; KAUST Computational Bioscience Research Center (CBRC), King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi ArabiaDepartment of Pathology, The First Affiliated Hospital of Harbin Medical University, 23 Youzheng Street, Nangang District, Harbin 150001, People’s Republic of ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Pathology, The First Affiliated Hospital of Harbin Medical University, 23 Youzheng Street, Nangang District, Harbin 150001, People’s Republic of China; Corresponding authorDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaSuffolk University, Boston, MA, USADepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, ChinaBiomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, VIC 3800, Australia; Monash Data Futures Institute, Monash University, Melbourne, VIC 3800, AustraliaComputer Science Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia; Corresponding authorSchool of Microelectronics, Southern University of Science and Technology, Shenzhen 518055, PR China; Corresponding authorDepartment of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China; Department of Neurosurgery, Shenzhen University General Hospital, Shenzhen, Guangdong Province 518100, China; Corresponding authorComputer Science Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia; KAUST Computational Bioscience Research Center (CBRC), King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia; Corresponding authorSummary: Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists’ annotation of region of interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) – based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.http://www.sciencedirect.com/science/article/pii/S2589004222021459PathologyCancerMachine learning |
| spellingShingle | Zhongxiao Li Yuwei Cong Xin Chen Jiping Qi Jingxian Sun Tao Yan He Yang Junsi Liu Enzhou Lu Lixiang Wang Jiafeng Li Hong Hu Cheng Zhang Quan Yang Jiawei Yao Penglei Yao Qiuyi Jiang Wenwu Liu Jiangning Song Lawrence Carin Yupeng Chen Shiguang Zhao Xin Gao Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors iScience Pathology Cancer Machine learning |
| title | Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors |
| title_full | Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors |
| title_fullStr | Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors |
| title_full_unstemmed | Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors |
| title_short | Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors |
| title_sort | vision transformer based weakly supervised histopathological image analysis of primary brain tumors |
| topic | Pathology Cancer Machine learning |
| url | http://www.sciencedirect.com/science/article/pii/S2589004222021459 |
| work_keys_str_mv | AT zhongxiaoli visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT yuweicong visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT xinchen visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT jipingqi visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT jingxiansun visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT taoyan visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT heyang visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT junsiliu visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT enzhoulu visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT lixiangwang visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT jiafengli visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT honghu visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT chengzhang visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT quanyang visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT jiaweiyao visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT pengleiyao visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT qiuyijiang visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT wenwuliu visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT jiangningsong visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT lawrencecarin visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT yupengchen visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT shiguangzhao visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors AT xingao visiontransformerbasedweaklysupervisedhistopathologicalimageanalysisofprimarybraintumors |