Predicting outcomes at the individual patient level: what is the best method?
Objective When developing prediction models, researchers commonly employ a single model which uses all the available data (end-to-end approach). Alternatively, a similarity-based approach has been previously proposed, in which patients with similar clinical characteristics are first grouped into clu...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2023-10-01
|
| Series: | BMJ Mental Health |
| Online Access: | https://ebmh.bmj.com/content/26/1/e300701.full |
| _version_ | 1797367890453725184 |
|---|---|
| author | Qiang Liu Orestis Efthimiou Edoardo Giuseppe Ostinelli Anneka Tomlinson Franco De Crescenzo Zhenpeng Li |
| author_facet | Qiang Liu Orestis Efthimiou Edoardo Giuseppe Ostinelli Anneka Tomlinson Franco De Crescenzo Zhenpeng Li |
| author_sort | Qiang Liu |
| collection | DOAJ |
| description | Objective When developing prediction models, researchers commonly employ a single model which uses all the available data (end-to-end approach). Alternatively, a similarity-based approach has been previously proposed, in which patients with similar clinical characteristics are first grouped into clusters, then prediction models are developed within each cluster. The potential advantage of the similarity-based approach is that it may better address heterogeneity in patient characteristics. However, it remains unclear whether it improves the overall predictive performance. We illustrate the similarity-based approach using data from people with depression and empirically compare its performance with the end-to-end approach.Methods We used primary care data collected in general practices in the UK. Using 31 predefined baseline variables, we aimed to predict the severity of depressive symptoms, measured by Patient Health Questionnaire-9, 60 days after initiation of antidepressant treatment. Following the similarity-based approach, we used k-means to cluster patients based on their baseline characteristics. We derived the optimal number of clusters using the Silhouette coefficient. We used ridge regression to build prediction models in both approaches. To compare the models’ performance, we calculated the mean absolute error (MAE) and the coefficient of determination (R2) using bootstrapping.Results We analysed data from 16 384 patients. The end-to-end approach resulted in an MAE of 4.64 and R2 of 0.20. The best-performing similarity-based model was for four clusters, with MAE of 4.65 and R2 of 0.19.Conclusions The end-to-end and the similarity-based model yielded comparable performance. Due to its simplicity, the end-to-end approach can be favoured when using demographic and clinical data to build prediction models on pharmacological treatments for depression. |
| first_indexed | 2024-03-08T17:24:38Z |
| format | Article |
| id | doaj.art-5cbba4e4c3ef48439c5a6bc05f615ce7 |
| institution | Directory Open Access Journal |
| issn | 2755-9734 |
| language | English |
| last_indexed | 2024-03-08T17:24:38Z |
| publishDate | 2023-10-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Mental Health |
| spelling | doaj.art-5cbba4e4c3ef48439c5a6bc05f615ce72024-01-02T23:30:07ZengBMJ Publishing GroupBMJ Mental Health2755-97342023-10-0126110.1136/bmjment-2023-300701Predicting outcomes at the individual patient level: what is the best method?Qiang Liu0Orestis Efthimiou1Edoardo Giuseppe Ostinelli2Anneka Tomlinson3Franco De Crescenzo4Zhenpeng Li5Linyi People`s Hospital, Linyi, Shandong, China2 Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK4 Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK1 Department of Psychiatry, University of Oxford, Oxford, Oxfordshire, UK1 Department of Psychiatry, University of Oxford, Oxford, UK1 Department of Psychiatry, University of Oxford, Oxford, UKObjective When developing prediction models, researchers commonly employ a single model which uses all the available data (end-to-end approach). Alternatively, a similarity-based approach has been previously proposed, in which patients with similar clinical characteristics are first grouped into clusters, then prediction models are developed within each cluster. The potential advantage of the similarity-based approach is that it may better address heterogeneity in patient characteristics. However, it remains unclear whether it improves the overall predictive performance. We illustrate the similarity-based approach using data from people with depression and empirically compare its performance with the end-to-end approach.Methods We used primary care data collected in general practices in the UK. Using 31 predefined baseline variables, we aimed to predict the severity of depressive symptoms, measured by Patient Health Questionnaire-9, 60 days after initiation of antidepressant treatment. Following the similarity-based approach, we used k-means to cluster patients based on their baseline characteristics. We derived the optimal number of clusters using the Silhouette coefficient. We used ridge regression to build prediction models in both approaches. To compare the models’ performance, we calculated the mean absolute error (MAE) and the coefficient of determination (R2) using bootstrapping.Results We analysed data from 16 384 patients. The end-to-end approach resulted in an MAE of 4.64 and R2 of 0.20. The best-performing similarity-based model was for four clusters, with MAE of 4.65 and R2 of 0.19.Conclusions The end-to-end and the similarity-based model yielded comparable performance. Due to its simplicity, the end-to-end approach can be favoured when using demographic and clinical data to build prediction models on pharmacological treatments for depression.https://ebmh.bmj.com/content/26/1/e300701.full |
| spellingShingle | Qiang Liu Orestis Efthimiou Edoardo Giuseppe Ostinelli Anneka Tomlinson Franco De Crescenzo Zhenpeng Li Predicting outcomes at the individual patient level: what is the best method? BMJ Mental Health |
| title | Predicting outcomes at the individual patient level: what is the best method? |
| title_full | Predicting outcomes at the individual patient level: what is the best method? |
| title_fullStr | Predicting outcomes at the individual patient level: what is the best method? |
| title_full_unstemmed | Predicting outcomes at the individual patient level: what is the best method? |
| title_short | Predicting outcomes at the individual patient level: what is the best method? |
| title_sort | predicting outcomes at the individual patient level what is the best method |
| url | https://ebmh.bmj.com/content/26/1/e300701.full |
| work_keys_str_mv | AT qiangliu predictingoutcomesattheindividualpatientlevelwhatisthebestmethod AT orestisefthimiou predictingoutcomesattheindividualpatientlevelwhatisthebestmethod AT edoardogiuseppeostinelli predictingoutcomesattheindividualpatientlevelwhatisthebestmethod AT annekatomlinson predictingoutcomesattheindividualpatientlevelwhatisthebestmethod AT francodecrescenzo predictingoutcomesattheindividualpatientlevelwhatisthebestmethod AT zhenpengli predictingoutcomesattheindividualpatientlevelwhatisthebestmethod |