Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg
Abstract Objectives Machine learning models can support an individualized approach in the choice of bDMARDs. We developed prediction models for 5 different bDMARDs using machine learning methods based on patient data derived from the Austrian Biologics Registry (BioReg). Methods Data from 1397 patie...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-02-01
|
| Series: | Arthritis Research & Therapy |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13075-024-03277-x |
| _version_ | 1797273680146857984 |
|---|---|
| author | Dubravka Ukalovic Burkhard F. Leeb Bernhard Rintelen Gabriela Eichbauer-Sturm Peter Spellitz Rudolf Puchner Manfred Herold Miriam Stetter Vera Ferincz Johannes Resch-Passini Jochen Zwerina Marcus Zimmermann-Rittereiser Ruth Fritsch-Stork |
| author_facet | Dubravka Ukalovic Burkhard F. Leeb Bernhard Rintelen Gabriela Eichbauer-Sturm Peter Spellitz Rudolf Puchner Manfred Herold Miriam Stetter Vera Ferincz Johannes Resch-Passini Jochen Zwerina Marcus Zimmermann-Rittereiser Ruth Fritsch-Stork |
| author_sort | Dubravka Ukalovic |
| collection | DOAJ |
| description | Abstract Objectives Machine learning models can support an individualized approach in the choice of bDMARDs. We developed prediction models for 5 different bDMARDs using machine learning methods based on patient data derived from the Austrian Biologics Registry (BioReg). Methods Data from 1397 patients and 19 variables with at least 100 treat-to-target (t2t) courses per drug were derived from the BioReg biologics registry. Different machine learning algorithms were trained to predict the risk of ineffectiveness for each bDMARD within the first 26 weeks. Cross-validation and hyperparameter optimization were applied to generate the best models. Model quality was assessed by area under the receiver operating characteristic (AUROC). Using explainable AI (XAI), risk-reducing and risk-increasing factors were extracted. Results The best models per drug achieved an AUROC score of the following: abatacept, 0.66 (95% CI, 0.54–0.78); adalimumab, 0.70 (95% CI, 0.68–0.74); certolizumab, 0.84 (95% CI, 0.79–0.89); etanercept, 0.68 (95% CI, 0.55–0.87); tocilizumab, 0.72 (95% CI, 0.69–0.77). The most risk-increasing variables were visual analytic scores (VAS) for abatacept and etanercept and co-therapy with glucocorticoids for adalimumab. Dosage was the most important variable for certolizumab and associated with a lower risk of non-response. Some variables, such as gender and rheumatoid factor (RF), showed opposite impacts depending on the bDMARD. Conclusion Ineffectiveness of biological drugs could be predicted with promising accuracy. Interestingly, individual parameters were found to be associated with drug responses in different directions, indicating highly complex interactions. Machine learning can be of help in the decision-process by disentangling these relations. |
| first_indexed | 2024-03-07T14:47:48Z |
| format | Article |
| id | doaj.art-913cee245e814dc6866d24e0caf37144 |
| institution | Directory Open Access Journal |
| issn | 1478-6362 |
| language | English |
| last_indexed | 2024-03-07T14:47:48Z |
| publishDate | 2024-02-01 |
| publisher | BMC |
| record_format | Article |
| series | Arthritis Research & Therapy |
| spelling | doaj.art-913cee245e814dc6866d24e0caf371442024-03-05T19:52:01ZengBMCArthritis Research & Therapy1478-63622024-02-0126111210.1186/s13075-024-03277-xPrediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioRegDubravka Ukalovic0Burkhard F. Leeb1Bernhard Rintelen2Gabriela Eichbauer-Sturm3Peter Spellitz4Rudolf Puchner5Manfred Herold6Miriam Stetter7Vera Ferincz8Johannes Resch-Passini9Jochen Zwerina10Marcus Zimmermann-Rittereiser11Ruth Fritsch-Stork12Siemens Healthcare GmbH, Computed TomographyRheumatological Practice, Private OfficeLower Austrian State Hospital Stockerau, 2nd Department of Medicine, Lower Austrian Competence Center for Rheumatology, Karl Landsteiner Institute for Clinical RheumatologyRheumatological Practice, Private OfficeRheuma-Center Wien-Oberlaa, Department of RheumatologyRheumatological Practice, Private OfficeDepartment of Internal Medicine II, Medical University of InnsbruckRheumatological Practice, Private OfficeDepartment of Internal Medicine, University Hospital St. PöltenRheuma-Center Wien-Oberlaa, Department of RheumatologyHanusch KrankenhausSiemens Healthcare GmbH, Digital & AutomationHealth Care Center Mariahilf of ÖGKAbstract Objectives Machine learning models can support an individualized approach in the choice of bDMARDs. We developed prediction models for 5 different bDMARDs using machine learning methods based on patient data derived from the Austrian Biologics Registry (BioReg). Methods Data from 1397 patients and 19 variables with at least 100 treat-to-target (t2t) courses per drug were derived from the BioReg biologics registry. Different machine learning algorithms were trained to predict the risk of ineffectiveness for each bDMARD within the first 26 weeks. Cross-validation and hyperparameter optimization were applied to generate the best models. Model quality was assessed by area under the receiver operating characteristic (AUROC). Using explainable AI (XAI), risk-reducing and risk-increasing factors were extracted. Results The best models per drug achieved an AUROC score of the following: abatacept, 0.66 (95% CI, 0.54–0.78); adalimumab, 0.70 (95% CI, 0.68–0.74); certolizumab, 0.84 (95% CI, 0.79–0.89); etanercept, 0.68 (95% CI, 0.55–0.87); tocilizumab, 0.72 (95% CI, 0.69–0.77). The most risk-increasing variables were visual analytic scores (VAS) for abatacept and etanercept and co-therapy with glucocorticoids for adalimumab. Dosage was the most important variable for certolizumab and associated with a lower risk of non-response. Some variables, such as gender and rheumatoid factor (RF), showed opposite impacts depending on the bDMARD. Conclusion Ineffectiveness of biological drugs could be predicted with promising accuracy. Interestingly, individual parameters were found to be associated with drug responses in different directions, indicating highly complex interactions. Machine learning can be of help in the decision-process by disentangling these relations.https://doi.org/10.1186/s13075-024-03277-xRheumatoid arthritisbDMARDMachine learningRoutinely collected dataDMARDs |
| spellingShingle | Dubravka Ukalovic Burkhard F. Leeb Bernhard Rintelen Gabriela Eichbauer-Sturm Peter Spellitz Rudolf Puchner Manfred Herold Miriam Stetter Vera Ferincz Johannes Resch-Passini Jochen Zwerina Marcus Zimmermann-Rittereiser Ruth Fritsch-Stork Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg Arthritis Research & Therapy Rheumatoid arthritis bDMARD Machine learning Routinely collected data DMARDs |
| title | Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg |
| title_full | Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg |
| title_fullStr | Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg |
| title_full_unstemmed | Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg |
| title_short | Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg |
| title_sort | prediction of ineffectiveness of biological drugs using machine learning and explainable ai methods data from the austrian biological registry bioreg |
| topic | Rheumatoid arthritis bDMARD Machine learning Routinely collected data DMARDs |
| url | https://doi.org/10.1186/s13075-024-03277-x |
| work_keys_str_mv | AT dubravkaukalovic predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT burkhardfleeb predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT bernhardrintelen predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT gabrielaeichbauersturm predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT peterspellitz predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT rudolfpuchner predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT manfredherold predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT miriamstetter predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT veraferincz predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT johannesreschpassini predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT jochenzwerina predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT marcuszimmermannrittereiser predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg AT ruthfritschstork predictionofineffectivenessofbiologicaldrugsusingmachinelearningandexplainableaimethodsdatafromtheaustrianbiologicalregistrybioreg |