A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models
Summary: Background: Surveillance is universally recommended for non-small cell lung cancer (NSCLC) patients treated with curative-intent radiotherapy. High-quality evidence to inform optimal surveillance strategies is lacking. Machine learning demonstrates promise in accurate outcome prediction fo...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-03-01
|
| Series: | EBioMedicine |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352396422000950 |
| _version_ | 1818910106961575936 |
|---|---|
| author | Sumeet Hindocha Thomas G. Charlton Kristofer Linton-Reid Benjamin Hunter Charleen Chan Merina Ahmed Emily J. Robinson Matthew Orton Shahreen Ahmad Fiona McDonald Imogen Locke Danielle Power Matthew Blackledge Richard W. Lee Eric O. Aboagye |
| author_facet | Sumeet Hindocha Thomas G. Charlton Kristofer Linton-Reid Benjamin Hunter Charleen Chan Merina Ahmed Emily J. Robinson Matthew Orton Shahreen Ahmad Fiona McDonald Imogen Locke Danielle Power Matthew Blackledge Richard W. Lee Eric O. Aboagye |
| author_sort | Sumeet Hindocha |
| collection | DOAJ |
| description | Summary: Background: Surveillance is universally recommended for non-small cell lung cancer (NSCLC) patients treated with curative-intent radiotherapy. High-quality evidence to inform optimal surveillance strategies is lacking. Machine learning demonstrates promise in accurate outcome prediction for a variety of health conditions. The purpose of this study was to utilise readily available patient, tumour, and treatment data to develop, validate and externally test machine learning models for predicting recurrence, recurrence-free survival (RFS) and overall survival (OS) at 2 years from treatment. Methods: A retrospective, multicentre study of patients receiving curative-intent radiotherapy for NSCLC was undertaken. A total of 657 patients from 5 hospitals were eligible for inclusion. Data pre-processing derived 34 features for predictive modelling. Combinations of 8 feature reduction methods and 10 machine learning classification algorithms were compared, producing risk-stratification models for predicting recurrence, RFS and OS. Models were compared with 10-fold cross validation and an external test set and benchmarked against TNM-stage and performance status. Youden Index was derived from validation set ROC curves to distinguish high and low risk groups and Kaplan-Meier analyses performed. Findings: Median follow-up time was 852 days. Parameters were well matched across training-validation and external test sets: Mean age was 73 and 71 respectively, and recurrence, RFS and OS rates at 2 years were 43% vs 34%, 54% vs 47% and 54% vs 47% respectively. The respective validation and test set AUCs were as follows: 1) RFS: 0·682 (0·575–0·788) and 0·681 (0·597–0·766), 2) Recurrence: 0·687 (0·582–0·793) and 0·722 (0·635–0·81), and 3) OS: 0·759 (0·663–0·855) and 0·717 (0·634–0·8). Our models were superior to TNM stage and performance status in predicting recurrence and OS. Interpretation: This robust and ready to use machine learning method, validated and externally tested, sets the stage for future clinical trials entailing quantitative personalised risk-stratification and surveillance following curative-intent radiotherapy for NSCLC. Funding: A full list of funding bodies that contributed to this study can be found in the Acknowledgements section. |
| first_indexed | 2024-12-19T22:37:32Z |
| format | Article |
| id | doaj.art-338d2d91deec420ba0ba2951aac1baea |
| institution | Directory Open Access Journal |
| issn | 2352-3964 |
| language | English |
| last_indexed | 2024-12-19T22:37:32Z |
| publishDate | 2022-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | EBioMedicine |
| spelling | doaj.art-338d2d91deec420ba0ba2951aac1baea2022-12-21T20:03:09ZengElsevierEBioMedicine2352-39642022-03-0177103911A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction modelsSumeet Hindocha0Thomas G. Charlton1Kristofer Linton-Reid2Benjamin Hunter3Charleen Chan4Merina Ahmed5Emily J. Robinson6Matthew Orton7Shahreen Ahmad8Fiona McDonald9Imogen Locke10Danielle Power11Matthew Blackledge12Richard W. Lee13Eric O. Aboagye14Lung Unit, The Royal Marsden NHS Foundation Trust, Fulham Road, London SW36JJ, UK; AI for Healthcare Centre for Doctoral Training, Imperial College London, Exhibition Road, London SW7 2BX, UK; Department of Clinical Oncology, Institute of Cancer Research NIHR Biomedical Research Centre, London, UK; Cancer Imaging Centre, Department of Surgery and Cancer, Imperial College London, Du Cane Road, London W12 0NN, UK; Early Diagnosis and Detection Centre, National Institute for Health Research (NIHR) Biomedical Research Centre at The Royal Marsden NHS Foundation Trust and the Institute of Cancer Research, LondonGuy's Cancer Centre, Guy's and St Thomas’ NHS Foundation Trust, Great Maze Pond, London SE19RT UKCancer Imaging Centre, Department of Surgery and Cancer, Imperial College London, Du Cane Road, London W12 0NN, UKLung Unit, The Royal Marsden NHS Foundation Trust, Fulham Road, London SW36JJ, UK; Department of Clinical Oncology, Institute of Cancer Research NIHR Biomedical Research Centre, London, UK; Cancer Imaging Centre, Department of Surgery and Cancer, Imperial College London, Du Cane Road, London W12 0NN, UK; Early Diagnosis and Detection Centre, National Institute for Health Research (NIHR) Biomedical Research Centre at The Royal Marsden NHS Foundation Trust and the Institute of Cancer Research, LondonDepartment of Clinical Oncology, Institute of Cancer Research NIHR Biomedical Research Centre, London, UKLung Unit, The Royal Marsden NHS Foundation Trust, Downs Road, Sutton SM25PT, UKClinical Trials Unit, Royal Marsden NHS Foundation Trust, Downs Road, Sutton SM25PT, UKArtificial Intelligence Imaging Hub, Royal Marsden NHS Foundation Trust, Downs Road, Sutton SM25PT, UKGuy's Cancer Centre, Guy's and St Thomas’ NHS Foundation Trust, Great Maze Pond, London SE19RT UKLung Unit, The Royal Marsden NHS Foundation Trust, Fulham Road, London SW36JJ, UK; Department of Clinical Oncology, Institute of Cancer Research NIHR Biomedical Research Centre, London, UKLung Unit, The Royal Marsden NHS Foundation Trust, Downs Road, Sutton SM25PT, UKDepartment of Clinical Oncology, Charing Cross Hospital, Fulham Palace Road, London W6 8RF, UKRadiotherapy and Imaging, Institute of Cancer Research, 123 Old Brompton Road, London SW7 3RP, UKLung Unit, The Royal Marsden NHS Foundation Trust, Fulham Road, London SW36JJ, UK; Early Diagnosis and Detection Centre, National Institute for Health Research (NIHR) Biomedical Research Centre at The Royal Marsden NHS Foundation Trust and the Institute of Cancer Research, London; National Heart and Lung Institute, Imperial College, London, UK; Corresponding author at: Lung Unit, The Royal Marsden NHS Foundation Trust, Fulham Road, London SW36JJ, UK.Department of Clinical Oncology, Institute of Cancer Research NIHR Biomedical Research Centre, London, UK; Corresponding author at: Lung Unit, The Royal Marsden NHS Foundation Trust, Fulham Road, London SW36JJ, UK.Summary: Background: Surveillance is universally recommended for non-small cell lung cancer (NSCLC) patients treated with curative-intent radiotherapy. High-quality evidence to inform optimal surveillance strategies is lacking. Machine learning demonstrates promise in accurate outcome prediction for a variety of health conditions. The purpose of this study was to utilise readily available patient, tumour, and treatment data to develop, validate and externally test machine learning models for predicting recurrence, recurrence-free survival (RFS) and overall survival (OS) at 2 years from treatment. Methods: A retrospective, multicentre study of patients receiving curative-intent radiotherapy for NSCLC was undertaken. A total of 657 patients from 5 hospitals were eligible for inclusion. Data pre-processing derived 34 features for predictive modelling. Combinations of 8 feature reduction methods and 10 machine learning classification algorithms were compared, producing risk-stratification models for predicting recurrence, RFS and OS. Models were compared with 10-fold cross validation and an external test set and benchmarked against TNM-stage and performance status. Youden Index was derived from validation set ROC curves to distinguish high and low risk groups and Kaplan-Meier analyses performed. Findings: Median follow-up time was 852 days. Parameters were well matched across training-validation and external test sets: Mean age was 73 and 71 respectively, and recurrence, RFS and OS rates at 2 years were 43% vs 34%, 54% vs 47% and 54% vs 47% respectively. The respective validation and test set AUCs were as follows: 1) RFS: 0·682 (0·575–0·788) and 0·681 (0·597–0·766), 2) Recurrence: 0·687 (0·582–0·793) and 0·722 (0·635–0·81), and 3) OS: 0·759 (0·663–0·855) and 0·717 (0·634–0·8). Our models were superior to TNM stage and performance status in predicting recurrence and OS. Interpretation: This robust and ready to use machine learning method, validated and externally tested, sets the stage for future clinical trials entailing quantitative personalised risk-stratification and surveillance following curative-intent radiotherapy for NSCLC. Funding: A full list of funding bodies that contributed to this study can be found in the Acknowledgements section.http://www.sciencedirect.com/science/article/pii/S2352396422000950Non-small cell lung cancerRadiotherapyMachine learningRecurrenceOverall survivalPrediction |
| spellingShingle | Sumeet Hindocha Thomas G. Charlton Kristofer Linton-Reid Benjamin Hunter Charleen Chan Merina Ahmed Emily J. Robinson Matthew Orton Shahreen Ahmad Fiona McDonald Imogen Locke Danielle Power Matthew Blackledge Richard W. Lee Eric O. Aboagye A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models EBioMedicine Non-small cell lung cancer Radiotherapy Machine learning Recurrence Overall survival Prediction |
| title | A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models |
| title_full | A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models |
| title_fullStr | A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models |
| title_full_unstemmed | A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models |
| title_short | A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models |
| title_sort | comparison of machine learning methods for predicting recurrence and death after curative intent radiotherapy for non small cell lung cancer development and validation of multivariable clinical prediction models |
| topic | Non-small cell lung cancer Radiotherapy Machine learning Recurrence Overall survival Prediction |
| url | http://www.sciencedirect.com/science/article/pii/S2352396422000950 |
| work_keys_str_mv | AT sumeethindocha acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT thomasgcharlton acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT kristoferlintonreid acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT benjaminhunter acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT charleenchan acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT merinaahmed acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT emilyjrobinson acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT mattheworton acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT shahreenahmad acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT fionamcdonald acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT imogenlocke acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT daniellepower acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT matthewblackledge acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT richardwlee acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT ericoaboagye acomparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT sumeethindocha comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT thomasgcharlton comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT kristoferlintonreid comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT benjaminhunter comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT charleenchan comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT merinaahmed comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT emilyjrobinson comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT mattheworton comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT shahreenahmad comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT fionamcdonald comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT imogenlocke comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT daniellepower comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT matthewblackledge comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT richardwlee comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels AT ericoaboagye comparisonofmachinelearningmethodsforpredictingrecurrenceanddeathaftercurativeintentradiotherapyfornonsmallcelllungcancerdevelopmentandvalidationofmultivariableclinicalpredictionmodels |