Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study
BackgroundRecent developments in artificial intelligence suggest that radiomics may represent a promising non-invasive biomarker to predict response to immune checkpoint inhibitors (ICIs). Nevertheless, validation of radiomics algorithms in independent cohorts remains a challenge due to variations i...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-07-01
|
| Series: | Frontiers in Oncology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fonc.2023.1196414/full |
| _version_ | 1827897137216618496 |
|---|---|
| author | Marion Tonneau Marion Tonneau Kim Phan Venkata S. K. Manem Venkata S. K. Manem Cecile Low-Kam Francis Dutil Suzanne Kazandjian Davy Vanderweyen Justin Panasci Julie Malo François Coulombe Andréanne Gagné Arielle Elkrief Arielle Elkrief Wiam Belkaïd Lisa Di Jorio Michele Orain Nicole Bouchard Thierry Muanza Thierry Muanza Frank J. Rybicki Kam Kafi David Huntsman Philippe Joubert Philippe Joubert Florent Chandelier Bertrand Routy Bertrand Routy |
| author_facet | Marion Tonneau Marion Tonneau Kim Phan Venkata S. K. Manem Venkata S. K. Manem Cecile Low-Kam Francis Dutil Suzanne Kazandjian Davy Vanderweyen Justin Panasci Julie Malo François Coulombe Andréanne Gagné Arielle Elkrief Arielle Elkrief Wiam Belkaïd Lisa Di Jorio Michele Orain Nicole Bouchard Thierry Muanza Thierry Muanza Frank J. Rybicki Kam Kafi David Huntsman Philippe Joubert Philippe Joubert Florent Chandelier Bertrand Routy Bertrand Routy |
| author_sort | Marion Tonneau |
| collection | DOAJ |
| description | BackgroundRecent developments in artificial intelligence suggest that radiomics may represent a promising non-invasive biomarker to predict response to immune checkpoint inhibitors (ICIs). Nevertheless, validation of radiomics algorithms in independent cohorts remains a challenge due to variations in image acquisition and reconstruction. Using radiomics, we investigated the importance of scan normalization as part of a broader machine learning framework to enable model external generalizability to predict ICI response in non-small cell lung cancer (NSCLC) patients across different centers.MethodsRadiomics features were extracted and compared from 642 advanced NSCLC patients on pre-ICI scans using established open-source PyRadiomics and a proprietary DeepRadiomics deep learning technology. The population was separated into two groups: a discovery cohort of 512 NSCLC patients from three academic centers and a validation cohort that included 130 NSCLC patients from a fourth center. We harmonized images to account for variations in reconstruction kernel, slice thicknesses, and device manufacturers. Multivariable models, evaluated using cross-validation, were used to estimate the predictive value of clinical variables, PD-L1 expression, and PyRadiomics or DeepRadiomics for progression-free survival at 6 months (PFS-6).ResultsThe best prognostic factor for PFS-6, excluding radiomics features, was obtained with the combination of Clinical + PD-L1 expression (AUC = 0.66 in the discovery and 0.62 in the validation cohort). Without image harmonization, combining Clinical + PyRadiomics or DeepRadiomics delivered an AUC = 0.69 and 0.69, respectively, in the discovery cohort, but dropped to 0.57 and 0.52, in the validation cohort. This lack of generalizability was consistent with observations in principal component analysis clustered by CT scan parameters. Subsequently, image harmonization eliminated these clusters. The combination of Clinical + DeepRadiomics reached an AUC = 0.67 and 0.63 in the discovery and validation cohort, respectively. Conversely, the combination of Clinical + PyRadiomics failed generalizability validations, with AUC = 0.66 and 0.59.ConclusionWe demonstrated that a risk prediction model combining Clinical + DeepRadiomics was generalizable following CT scan harmonization and machine learning generalization methods. These results had similar performances to routine oncology practice using Clinical + PD-L1. This study supports the strong potential of radiomics as a future non-invasive strategy to predict ICI response in advanced NSCLC. |
| first_indexed | 2024-03-12T22:44:53Z |
| format | Article |
| id | doaj.art-1c9b27366e80483d88227a18e19ca975 |
| institution | Directory Open Access Journal |
| issn | 2234-943X |
| language | English |
| last_indexed | 2024-03-12T22:44:53Z |
| publishDate | 2023-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Oncology |
| spelling | doaj.art-1c9b27366e80483d88227a18e19ca9752023-07-21T07:49:45ZengFrontiers Media S.A.Frontiers in Oncology2234-943X2023-07-011310.3389/fonc.2023.11964141196414Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort studyMarion Tonneau0Marion Tonneau1Kim Phan2Venkata S. K. Manem3Venkata S. K. Manem4Cecile Low-Kam5Francis Dutil6Suzanne Kazandjian7Davy Vanderweyen8Justin Panasci9Julie Malo10François Coulombe11Andréanne Gagné12Arielle Elkrief13Arielle Elkrief14Wiam Belkaïd15Lisa Di Jorio16Michele Orain17Nicole Bouchard18Thierry Muanza19Thierry Muanza20Frank J. Rybicki21Kam Kafi22David Huntsman23Philippe Joubert24Philippe Joubert25Florent Chandelier26Bertrand Routy27Bertrand Routy28Department of Cancer Research, Centre de Recherche du Centre Hospitalier Universitaire de Montréal (CRCHUM), Montreal, QC, CanadaUniversité de Médecine, Lille, FranceImagia Canexia Health, Montreal, QC, CanadaInstitut Universitaire de Cardiologie et de Pneumologie de Quebec, Université Laval, Québec City, QC, CanadaDepartment of Mathematics and Computer Science, University of Quebec at Trois-Rivières, Trois-Rivières, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaDepartment of Medical Oncology, Jewish General Hospital, Montreal, QC, CanadaDepartment of Radiology, Centre Hospitalier de Sherbrooke (CHUS), Sherbrooke, QC, CanadaDepartment of Medical Oncology, Jewish General Hospital, Montreal, QC, CanadaDepartment of Cancer Research, Centre de Recherche du Centre Hospitalier Universitaire de Montréal (CRCHUM), Montreal, QC, CanadaInstitut Universitaire de Cardiologie et de Pneumologie de Quebec, Université Laval, Québec City, QC, CanadaInstitut Universitaire de Cardiologie et de Pneumologie de Quebec, Université Laval, Québec City, QC, CanadaDepartment of Cancer Research, Centre de Recherche du Centre Hospitalier Universitaire de Montréal (CRCHUM), Montreal, QC, CanadaHemato-Oncology Division, Centre Hospitalier de l’université de Montreal, Montreal, QC, CanadaDepartment of Cancer Research, Centre de Recherche du Centre Hospitalier Universitaire de Montréal (CRCHUM), Montreal, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaInstitut Universitaire de Cardiologie et de Pneumologie de Quebec, Université Laval, Québec City, QC, CanadaDepartment of Oncology, Centre Hospitalier de Sherbrooke (CHUS), Sherbrooke, QC, CanadaDepartment of Medical Oncology, Jewish General Hospital, Montreal, QC, Canada0Department of Radiation Oncology, Lady Davis Institute of the Jewish General Hospital, Montreal, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaInstitut Universitaire de Cardiologie et de Pneumologie de Quebec, Université Laval, Québec City, QC, Canada1Department of Pathology, Institut Universitaire de Cardiologie et de Pneumologie de Québec, Québec, QC, CanadaImagia Canexia Health, Montreal, QC, CanadaDepartment of Cancer Research, Centre de Recherche du Centre Hospitalier Universitaire de Montréal (CRCHUM), Montreal, QC, CanadaHemato-Oncology Division, Centre Hospitalier de l’université de Montreal, Montreal, QC, CanadaBackgroundRecent developments in artificial intelligence suggest that radiomics may represent a promising non-invasive biomarker to predict response to immune checkpoint inhibitors (ICIs). Nevertheless, validation of radiomics algorithms in independent cohorts remains a challenge due to variations in image acquisition and reconstruction. Using radiomics, we investigated the importance of scan normalization as part of a broader machine learning framework to enable model external generalizability to predict ICI response in non-small cell lung cancer (NSCLC) patients across different centers.MethodsRadiomics features were extracted and compared from 642 advanced NSCLC patients on pre-ICI scans using established open-source PyRadiomics and a proprietary DeepRadiomics deep learning technology. The population was separated into two groups: a discovery cohort of 512 NSCLC patients from three academic centers and a validation cohort that included 130 NSCLC patients from a fourth center. We harmonized images to account for variations in reconstruction kernel, slice thicknesses, and device manufacturers. Multivariable models, evaluated using cross-validation, were used to estimate the predictive value of clinical variables, PD-L1 expression, and PyRadiomics or DeepRadiomics for progression-free survival at 6 months (PFS-6).ResultsThe best prognostic factor for PFS-6, excluding radiomics features, was obtained with the combination of Clinical + PD-L1 expression (AUC = 0.66 in the discovery and 0.62 in the validation cohort). Without image harmonization, combining Clinical + PyRadiomics or DeepRadiomics delivered an AUC = 0.69 and 0.69, respectively, in the discovery cohort, but dropped to 0.57 and 0.52, in the validation cohort. This lack of generalizability was consistent with observations in principal component analysis clustered by CT scan parameters. Subsequently, image harmonization eliminated these clusters. The combination of Clinical + DeepRadiomics reached an AUC = 0.67 and 0.63 in the discovery and validation cohort, respectively. Conversely, the combination of Clinical + PyRadiomics failed generalizability validations, with AUC = 0.66 and 0.59.ConclusionWe demonstrated that a risk prediction model combining Clinical + DeepRadiomics was generalizable following CT scan harmonization and machine learning generalization methods. These results had similar performances to routine oncology practice using Clinical + PD-L1. This study supports the strong potential of radiomics as a future non-invasive strategy to predict ICI response in advanced NSCLC.https://www.frontiersin.org/articles/10.3389/fonc.2023.1196414/fullradiomicsDeeplearningNSCLCimmunotherapyDeepRadiomics |
| spellingShingle | Marion Tonneau Marion Tonneau Kim Phan Venkata S. K. Manem Venkata S. K. Manem Cecile Low-Kam Francis Dutil Suzanne Kazandjian Davy Vanderweyen Justin Panasci Julie Malo François Coulombe Andréanne Gagné Arielle Elkrief Arielle Elkrief Wiam Belkaïd Lisa Di Jorio Michele Orain Nicole Bouchard Thierry Muanza Thierry Muanza Frank J. Rybicki Kam Kafi David Huntsman Philippe Joubert Philippe Joubert Florent Chandelier Bertrand Routy Bertrand Routy Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study Frontiers in Oncology radiomics Deeplearning NSCLC immunotherapy DeepRadiomics |
| title | Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study |
| title_full | Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study |
| title_fullStr | Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study |
| title_full_unstemmed | Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study |
| title_short | Generalization optimizing machine learning to improve CT scan radiomics and assess immune checkpoint inhibitors’ response in non-small cell lung cancer: a multicenter cohort study |
| title_sort | generalization optimizing machine learning to improve ct scan radiomics and assess immune checkpoint inhibitors response in non small cell lung cancer a multicenter cohort study |
| topic | radiomics Deeplearning NSCLC immunotherapy DeepRadiomics |
| url | https://www.frontiersin.org/articles/10.3389/fonc.2023.1196414/full |
| work_keys_str_mv | AT mariontonneau generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT mariontonneau generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT kimphan generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT venkataskmanem generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT venkataskmanem generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT cecilelowkam generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT francisdutil generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT suzannekazandjian generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT davyvanderweyen generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT justinpanasci generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT juliemalo generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT francoiscoulombe generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT andreannegagne generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT arielleelkrief generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT arielleelkrief generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT wiambelkaid generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT lisadijorio generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT micheleorain generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT nicolebouchard generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT thierrymuanza generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT thierrymuanza generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT frankjrybicki generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT kamkafi generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT davidhuntsman generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT philippejoubert generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT philippejoubert generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT florentchandelier generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT bertrandrouty generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy AT bertrandrouty generalizationoptimizingmachinelearningtoimprovectscanradiomicsandassessimmunecheckpointinhibitorsresponseinnonsmallcelllungcanceramulticentercohortstudy |