Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques
Abstract Background Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2020-11-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12874-020-01153-1 |
| _version_ | 1828780489765289984 |
|---|---|
| author | Georgios Kantidakis Hein Putter Carlo Lancia Jacob de Boer Andries E. Braat Marta Fiocco |
| author_facet | Georgios Kantidakis Hein Putter Carlo Lancia Jacob de Boer Andries E. Braat Marta Fiocco |
| author_sort | Georgios Kantidakis |
| collection | DOAJ |
| description | Abstract Background Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians. Methods In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques. Results Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years. Conclusion In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables. Trial registration Retrospective data were provided by the Scientific Registry of Transplant Recipients under Data Use Agreement number 9477 for analysis of risk factors after liver transplantation. |
| first_indexed | 2024-12-11T17:18:52Z |
| format | Article |
| id | doaj.art-3dd6a3ea545f40d8b861cf6d60fb4337 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-11T17:18:52Z |
| publishDate | 2020-11-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-3dd6a3ea545f40d8b861cf6d60fb43372022-12-22T00:57:14ZengBMCBMC Medical Research Methodology1471-22882020-11-0120111410.1186/s12874-020-01153-1Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniquesGeorgios Kantidakis0Hein Putter1Carlo Lancia2Jacob de Boer3Andries E. Braat4Marta Fiocco5Mathematical Institute (MI) Leiden UniversityDepartment of Biomedical Data Sciences, Section Medical Statistics, Leiden University Medical Center (LUMC)Mathematical Institute (MI) Leiden UniversityDepartment of Surgery, Leiden University Medical Center (LUMC)Department of Surgery, Leiden University Medical Center (LUMC)Mathematical Institute (MI) Leiden UniversityAbstract Background Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians. Methods In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques. Results Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years. Conclusion In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables. Trial registration Retrospective data were provided by the Scientific Registry of Transplant Recipients under Data Use Agreement number 9477 for analysis of risk factors after liver transplantation.http://link.springer.com/article/10.1186/s12874-020-01153-1Random survival forestNeural networksPredictive performanceRisk factorsPost-transplantationSurvival analysis |
| spellingShingle | Georgios Kantidakis Hein Putter Carlo Lancia Jacob de Boer Andries E. Braat Marta Fiocco Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques BMC Medical Research Methodology Random survival forest Neural networks Predictive performance Risk factors Post-transplantation Survival analysis |
| title | Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques |
| title_full | Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques |
| title_fullStr | Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques |
| title_full_unstemmed | Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques |
| title_short | Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques |
| title_sort | survival prediction models since liver transplantation comparisons between cox models and machine learning techniques |
| topic | Random survival forest Neural networks Predictive performance Risk factors Post-transplantation Survival analysis |
| url | http://link.springer.com/article/10.1186/s12874-020-01153-1 |
| work_keys_str_mv | AT georgioskantidakis survivalpredictionmodelssincelivertransplantationcomparisonsbetweencoxmodelsandmachinelearningtechniques AT heinputter survivalpredictionmodelssincelivertransplantationcomparisonsbetweencoxmodelsandmachinelearningtechniques AT carlolancia survivalpredictionmodelssincelivertransplantationcomparisonsbetweencoxmodelsandmachinelearningtechniques AT jacobdeboer survivalpredictionmodelssincelivertransplantationcomparisonsbetweencoxmodelsandmachinelearningtechniques AT andriesebraat survivalpredictionmodelssincelivertransplantationcomparisonsbetweencoxmodelsandmachinelearningtechniques AT martafiocco survivalpredictionmodelssincelivertransplantationcomparisonsbetweencoxmodelsandmachinelearningtechniques |