A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy
BackgroundMachine learning (ML) algorithms are increasingly explored in glioma prognostication. Random survival forest (RSF) is a common ML approach in analyzing time-to-event survival data. However, it is controversial which method between RSF and traditional cornerstone method Cox proportional haz...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2020-10-01
|
| Series: | Frontiers in Oncology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fonc.2020.551420/full |
| _version_ | 1817982745386680320 |
|---|---|
| author | Xianxin Qiu Xianxin Qiu Jing Gao Jing Gao Jing Yang Jing Yang Jiyi Hu Jiyi Hu Weixu Hu Weixu Hu Lin Kong Lin Kong Jiade J. Lu Jiade J. Lu |
| author_facet | Xianxin Qiu Xianxin Qiu Jing Gao Jing Gao Jing Yang Jing Yang Jiyi Hu Jiyi Hu Weixu Hu Weixu Hu Lin Kong Lin Kong Jiade J. Lu Jiade J. Lu |
| author_sort | Xianxin Qiu |
| collection | DOAJ |
| description | BackgroundMachine learning (ML) algorithms are increasingly explored in glioma prognostication. Random survival forest (RSF) is a common ML approach in analyzing time-to-event survival data. However, it is controversial which method between RSF and traditional cornerstone method Cox proportional hazards (CPH) is better fitted. The purpose of this study was to compare RSF and CPH in predicting tumor progression of high-grade glioma (HGG) after particle beam radiotherapy (PBRT).MethodsThe study enrolled 82 consecutive HGG patients who were treated with PBRT at Shanghai Proton and Heavy Ion Center between 6/2015 and 11/2019. The entire cohort was split into the training and testing set in an 80/20 ratio. Ten variables from patient-related, tumor-related and treatment-related information were utilized for developing CPH and RSF for predicting progression-free survival (PFS). The model performance was compared in concordance index (C-index) for discrimination (accuracy), brier score (BS) for calibration (precision) and variable importance for interpretability.ResultsThe CPH model demonstrated a better performance in terms of integrated C-index (62.9%) and BS (0.159) compared to RSF model (C-index = 61.1%, BS = 0.174). In the context of variable importance, CPH model indicated that age (P = 0.024), WHO grade (P = 0.020), IDH gene (P = 0.019), and MGMT promoter status (P = 0.040) were significantly correlated with PFS in the univariate analysis; multivariate analysis showed that age (P = 0.041), surgical completeness (P = 0.084), IDH gene (P = 0.057), and MGMT promoter (P = 0.092) had a significant or trend toward the relation with PFS. RSF showed that merely IDH and age were of positive importance for predicting PFS. A final nomogram was developed to predict tumor progression at the individual level based on CPH model.ConclusionsIn a relatively small dataset with HGG patients treated with PBRT, CPH outperformed RSF for predicting tumor progression. A comprehensive criterion with accuracy, precision, and interpretability is recommended in evaluating ML prognostication approaches for clinical deployment. |
| first_indexed | 2024-04-13T23:24:32Z |
| format | Article |
| id | doaj.art-9ad2806895674afba20c312ffae845d7 |
| institution | Directory Open Access Journal |
| issn | 2234-943X |
| language | English |
| last_indexed | 2024-04-13T23:24:32Z |
| publishDate | 2020-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Oncology |
| spelling | doaj.art-9ad2806895674afba20c312ffae845d72022-12-22T02:25:07ZengFrontiers Media S.A.Frontiers in Oncology2234-943X2020-10-011010.3389/fonc.2020.551420551420A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion RadiotherapyXianxin Qiu0Xianxin Qiu1Jing Gao2Jing Gao3Jing Yang4Jing Yang5Jiyi Hu6Jiyi Hu7Weixu Hu8Weixu Hu9Lin Kong10Lin Kong11Jiade J. Lu12Jiade J. Lu13Shanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Shanghai, ChinaShanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Shanghai, ChinaShanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Shanghai, ChinaShanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Shanghai, ChinaShanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Shanghai, ChinaShanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Fudan University Cancer Center, Shanghai, ChinaShanghai Engineering Research Center of Proton and Heavy Ion Radiation Therapy, Shanghai, ChinaDepartment of Radiation Oncology, Shanghai Proton and Heavy Ion Center, Shanghai, ChinaBackgroundMachine learning (ML) algorithms are increasingly explored in glioma prognostication. Random survival forest (RSF) is a common ML approach in analyzing time-to-event survival data. However, it is controversial which method between RSF and traditional cornerstone method Cox proportional hazards (CPH) is better fitted. The purpose of this study was to compare RSF and CPH in predicting tumor progression of high-grade glioma (HGG) after particle beam radiotherapy (PBRT).MethodsThe study enrolled 82 consecutive HGG patients who were treated with PBRT at Shanghai Proton and Heavy Ion Center between 6/2015 and 11/2019. The entire cohort was split into the training and testing set in an 80/20 ratio. Ten variables from patient-related, tumor-related and treatment-related information were utilized for developing CPH and RSF for predicting progression-free survival (PFS). The model performance was compared in concordance index (C-index) for discrimination (accuracy), brier score (BS) for calibration (precision) and variable importance for interpretability.ResultsThe CPH model demonstrated a better performance in terms of integrated C-index (62.9%) and BS (0.159) compared to RSF model (C-index = 61.1%, BS = 0.174). In the context of variable importance, CPH model indicated that age (P = 0.024), WHO grade (P = 0.020), IDH gene (P = 0.019), and MGMT promoter status (P = 0.040) were significantly correlated with PFS in the univariate analysis; multivariate analysis showed that age (P = 0.041), surgical completeness (P = 0.084), IDH gene (P = 0.057), and MGMT promoter (P = 0.092) had a significant or trend toward the relation with PFS. RSF showed that merely IDH and age were of positive importance for predicting PFS. A final nomogram was developed to predict tumor progression at the individual level based on CPH model.ConclusionsIn a relatively small dataset with HGG patients treated with PBRT, CPH outperformed RSF for predicting tumor progression. A comprehensive criterion with accuracy, precision, and interpretability is recommended in evaluating ML prognostication approaches for clinical deployment.https://www.frontiersin.org/articles/10.3389/fonc.2020.551420/fullhigh-grade gliomarandom survival forestmachine learningparticle beam radiotherapypredictive analytics |
| spellingShingle | Xianxin Qiu Xianxin Qiu Jing Gao Jing Gao Jing Yang Jing Yang Jiyi Hu Jiyi Hu Weixu Hu Weixu Hu Lin Kong Lin Kong Jiade J. Lu Jiade J. Lu A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy Frontiers in Oncology high-grade glioma random survival forest machine learning particle beam radiotherapy predictive analytics |
| title | A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy |
| title_full | A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy |
| title_fullStr | A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy |
| title_full_unstemmed | A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy |
| title_short | A Comparison Study of Machine Learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy |
| title_sort | comparison study of machine learning random survival forest and classic statistic cox proportional hazards for predicting progression in high grade glioma after proton and carbon ion radiotherapy |
| topic | high-grade glioma random survival forest machine learning particle beam radiotherapy predictive analytics |
| url | https://www.frontiersin.org/articles/10.3389/fonc.2020.551420/full |
| work_keys_str_mv | AT xianxinqiu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT xianxinqiu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jinggao acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jinggao acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jingyang acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jingyang acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiyihu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiyihu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT weixuhu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT weixuhu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT linkong acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT linkong acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiadejlu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiadejlu acomparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT xianxinqiu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT xianxinqiu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jinggao comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jinggao comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jingyang comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jingyang comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiyihu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiyihu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT weixuhu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT weixuhu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT linkong comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT linkong comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiadejlu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy AT jiadejlu comparisonstudyofmachinelearningrandomsurvivalforestandclassicstatisticcoxproportionalhazardsforpredictingprogressioninhighgradegliomaafterprotonandcarbonionradiotherapy |