A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer
Lung cancer accounts for more deaths worldwide than any other cancer disease. In order to provide patients with the most effective treatment for these aggressive tumours, multimodal learning is emerging as a new and promising field of research that aims to extract complementary information from the...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/8/11/298 |
| _version_ | 1827646437240864768 |
|---|---|
| author | Camillo Maria Caruso Valerio Guarrasi Ermanno Cordelli Rosa Sicilia Silvia Gentile Laura Messina Michele Fiore Claudia Piccolo Bruno Beomonte Zobel Giulio Iannello Sara Ramella Paolo Soda |
| author_facet | Camillo Maria Caruso Valerio Guarrasi Ermanno Cordelli Rosa Sicilia Silvia Gentile Laura Messina Michele Fiore Claudia Piccolo Bruno Beomonte Zobel Giulio Iannello Sara Ramella Paolo Soda |
| author_sort | Camillo Maria Caruso |
| collection | DOAJ |
| description | Lung cancer accounts for more deaths worldwide than any other cancer disease. In order to provide patients with the most effective treatment for these aggressive tumours, multimodal learning is emerging as a new and promising field of research that aims to extract complementary information from the data of different modalities for prognostic and predictive purposes. This knowledge could be used to optimise current treatments and maximise their effectiveness. To predict overall survival, in this work, we investigate the use of multimodal learning on the CLARO dataset, which includes CT images and clinical data collected from a cohort of non-small-cell lung cancer patients. Our method allows the identification of the optimal set of classifiers to be included in the ensemble in a late fusion approach. Specifically, after training unimodal models on each modality, it selects the best ensemble by solving a multiobjective optimisation problem that maximises both the recognition performance and the diversity of the predictions. In the ensemble, the labels of each sample are assigned using the majority voting rule. As further validation, we show that the proposed ensemble outperforms the models learning a single modality, obtaining state-of-the-art results on the task at hand. |
| first_indexed | 2024-03-09T18:57:53Z |
| format | Article |
| id | doaj.art-9f43983c043040b2b41a7f20b0715321 |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-09T18:57:53Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-9f43983c043040b2b41a7f20b07153212023-11-24T05:20:51ZengMDPI AGJournal of Imaging2313-433X2022-11-0181129810.3390/jimaging8110298A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung CancerCamillo Maria Caruso0Valerio Guarrasi1Ermanno Cordelli2Rosa Sicilia3Silvia Gentile4Laura Messina5Michele Fiore6Claudia Piccolo7Bruno Beomonte Zobel8Giulio Iannello9Sara Ramella10Paolo Soda11Research Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, ItalyResearch Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, ItalyResearch Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, ItalyResearch Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, ItalyOperative Research Unit of Radiation Oncology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, ItalyOperative Research Unit of Diagnostic Imaging, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, ItalyOperative Research Unit of Radiation Oncology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, ItalyOperative Research Unit of Diagnostic Imaging, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, ItalyOperative Research Unit of Diagnostic Imaging, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, ItalyResearch Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, ItalyOperative Research Unit of Radiation Oncology, Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, ItalyResearch Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, ItalyLung cancer accounts for more deaths worldwide than any other cancer disease. In order to provide patients with the most effective treatment for these aggressive tumours, multimodal learning is emerging as a new and promising field of research that aims to extract complementary information from the data of different modalities for prognostic and predictive purposes. This knowledge could be used to optimise current treatments and maximise their effectiveness. To predict overall survival, in this work, we investigate the use of multimodal learning on the CLARO dataset, which includes CT images and clinical data collected from a cohort of non-small-cell lung cancer patients. Our method allows the identification of the optimal set of classifiers to be included in the ensemble in a late fusion approach. Specifically, after training unimodal models on each modality, it selects the best ensemble by solving a multiobjective optimisation problem that maximises both the recognition performance and the diversity of the predictions. In the ensemble, the labels of each sample are assigned using the majority voting rule. As further validation, we show that the proposed ensemble outperforms the models learning a single modality, obtaining state-of-the-art results on the task at hand.https://www.mdpi.com/2313-433X/8/11/298multimodal deep learningmultiexpert systemsoptimisationconvolutional neural networksprecision medicineoncology |
| spellingShingle | Camillo Maria Caruso Valerio Guarrasi Ermanno Cordelli Rosa Sicilia Silvia Gentile Laura Messina Michele Fiore Claudia Piccolo Bruno Beomonte Zobel Giulio Iannello Sara Ramella Paolo Soda A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer Journal of Imaging multimodal deep learning multiexpert systems optimisation convolutional neural networks precision medicine oncology |
| title | A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer |
| title_full | A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer |
| title_fullStr | A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer |
| title_full_unstemmed | A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer |
| title_short | A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer |
| title_sort | multimodal ensemble driven by multiobjective optimisation to predict overall survival in non small cell lung cancer |
| topic | multimodal deep learning multiexpert systems optimisation convolutional neural networks precision medicine oncology |
| url | https://www.mdpi.com/2313-433X/8/11/298 |
| work_keys_str_mv | AT camillomariacaruso amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT valerioguarrasi amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT ermannocordelli amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT rosasicilia amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT silviagentile amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT lauramessina amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT michelefiore amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT claudiapiccolo amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT brunobeomontezobel amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT giulioiannello amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT sararamella amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT paolosoda amultimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT camillomariacaruso multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT valerioguarrasi multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT ermannocordelli multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT rosasicilia multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT silviagentile multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT lauramessina multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT michelefiore multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT claudiapiccolo multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT brunobeomontezobel multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT giulioiannello multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT sararamella multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer AT paolosoda multimodalensembledrivenbymultiobjectiveoptimisationtopredictoverallsurvivalinnonsmallcelllungcancer |