Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture
Abstract Background Comparative effectiveness research (CER) using observational databases has been suggested to obtain personalized evidence of treatment effectiveness. Inferential difficulties remain using traditional CER approaches especially related to designating patients to reference classes a...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-07-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-022-01663-0 |
| _version_ | 1818198230343614464 |
|---|---|
| author | John M. Brooks Cole G. Chapman Sarah B. Floyd Brian K. Chen Charles A. Thigpen Michael Kissenberth |
| author_facet | John M. Brooks Cole G. Chapman Sarah B. Floyd Brian K. Chen Charles A. Thigpen Michael Kissenberth |
| author_sort | John M. Brooks |
| collection | DOAJ |
| description | Abstract Background Comparative effectiveness research (CER) using observational databases has been suggested to obtain personalized evidence of treatment effectiveness. Inferential difficulties remain using traditional CER approaches especially related to designating patients to reference classes a priori. A novel Instrumental Variable Causal Forest Algorithm (IV-CFA) has the potential to provide personalized evidence using observational data without designating reference classes a priori, but the consistency of the evidence when varying key algorithm parameters remains unclear. We investigated the consistency of IV-CFA estimates through application to a database of Medicare beneficiaries with proximal humerus fractures (PHFs) that previously revealed heterogeneity in the effects of early surgery using instrumental variable estimators. Methods IV-CFA was used to estimate patient-specific early surgery effects on both beneficial and detrimental outcomes using different combinations of algorithm parameters and estimate variation was assessed for a population of 72,751 fee-for-service Medicare beneficiaries with PHFs in 2011. Classification and regression trees (CART) were applied to these estimates to create ex-post reference classes and the consistency of these classes were assessed. Two-stage least squares (2SLS) estimators were applied to representative ex-post reference classes to scrutinize the estimates relative to known 2SLS properties. Results IV-CFA uncovered substantial early surgery effect heterogeneity across PHF patients, but estimates for individual patients varied with algorithm parameters. CART applied to these estimates revealed ex-post reference classes consistent across algorithm parameters. 2SLS estimates showed that ex-post reference classes containing older, frailer patients with more comorbidities, and lower utilizers of healthcare were less likely to benefit and more likely to have detriments from higher rates of early surgery. Conclusions IV-CFA provides an illuminating method to uncover ex-post reference classes of patients based on treatment effects using observational data with a strong instrumental variable. Interpretation of treatment effect estimates within each ex-post reference class using traditional CER methods remains conditional on the extent of measured information in the data. |
| first_indexed | 2024-12-12T02:02:34Z |
| format | Article |
| id | doaj.art-5ec09d8b7a8e47658f574cb3d9c863a1 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-12T02:02:34Z |
| publishDate | 2022-07-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-5ec09d8b7a8e47658f574cb3d9c863a12022-12-22T00:42:07ZengBMCBMC Medical Research Methodology1471-22882022-07-0122111610.1186/s12874-022-01663-0Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fractureJohn M. Brooks0Cole G. Chapman1Sarah B. Floyd2Brian K. Chen3Charles A. Thigpen4Michael Kissenberth5Center for Effectiveness Research in Orthopaedics - Arnold School of Public Health GreenvilleDepartment of Pharmacy Practice and Science, University of IowaCenter for Effectiveness Research in OrthopaedicsHealth Services Policy & Management, University of South Carolina Arnold School of Public HealthCenter for Effectiveness Research in OrthopaedicsCenter for Effectiveness Research in OrthopaedicsAbstract Background Comparative effectiveness research (CER) using observational databases has been suggested to obtain personalized evidence of treatment effectiveness. Inferential difficulties remain using traditional CER approaches especially related to designating patients to reference classes a priori. A novel Instrumental Variable Causal Forest Algorithm (IV-CFA) has the potential to provide personalized evidence using observational data without designating reference classes a priori, but the consistency of the evidence when varying key algorithm parameters remains unclear. We investigated the consistency of IV-CFA estimates through application to a database of Medicare beneficiaries with proximal humerus fractures (PHFs) that previously revealed heterogeneity in the effects of early surgery using instrumental variable estimators. Methods IV-CFA was used to estimate patient-specific early surgery effects on both beneficial and detrimental outcomes using different combinations of algorithm parameters and estimate variation was assessed for a population of 72,751 fee-for-service Medicare beneficiaries with PHFs in 2011. Classification and regression trees (CART) were applied to these estimates to create ex-post reference classes and the consistency of these classes were assessed. Two-stage least squares (2SLS) estimators were applied to representative ex-post reference classes to scrutinize the estimates relative to known 2SLS properties. Results IV-CFA uncovered substantial early surgery effect heterogeneity across PHF patients, but estimates for individual patients varied with algorithm parameters. CART applied to these estimates revealed ex-post reference classes consistent across algorithm parameters. 2SLS estimates showed that ex-post reference classes containing older, frailer patients with more comorbidities, and lower utilizers of healthcare were less likely to benefit and more likely to have detriments from higher rates of early surgery. Conclusions IV-CFA provides an illuminating method to uncover ex-post reference classes of patients based on treatment effects using observational data with a strong instrumental variable. Interpretation of treatment effect estimates within each ex-post reference class using traditional CER methods remains conditional on the extent of measured information in the data.https://doi.org/10.1186/s12874-022-01663-0Instrumental Variable Causal Forest AlgorithmClassification and regression trees (CART)Two-stage least squares (2SLS) estimatorsProximal humerus fractureSurgery |
| spellingShingle | John M. Brooks Cole G. Chapman Sarah B. Floyd Brian K. Chen Charles A. Thigpen Michael Kissenberth Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture BMC Medical Research Methodology Instrumental Variable Causal Forest Algorithm Classification and regression trees (CART) Two-stage least squares (2SLS) estimators Proximal humerus fracture Surgery |
| title | Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture |
| title_full | Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture |
| title_fullStr | Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture |
| title_full_unstemmed | Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture |
| title_short | Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture |
| title_sort | assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data the case of early surgery for shoulder fracture |
| topic | Instrumental Variable Causal Forest Algorithm Classification and regression trees (CART) Two-stage least squares (2SLS) estimators Proximal humerus fracture Surgery |
| url | https://doi.org/10.1186/s12874-022-01663-0 |
| work_keys_str_mv | AT johnmbrooks assessingtheabilityofaninstrumentalvariablecausalforestalgorithmtopersonalizetreatmentevidenceusingobservationaldatathecaseofearlysurgeryforshoulderfracture AT colegchapman assessingtheabilityofaninstrumentalvariablecausalforestalgorithmtopersonalizetreatmentevidenceusingobservationaldatathecaseofearlysurgeryforshoulderfracture AT sarahbfloyd assessingtheabilityofaninstrumentalvariablecausalforestalgorithmtopersonalizetreatmentevidenceusingobservationaldatathecaseofearlysurgeryforshoulderfracture AT briankchen assessingtheabilityofaninstrumentalvariablecausalforestalgorithmtopersonalizetreatmentevidenceusingobservationaldatathecaseofearlysurgeryforshoulderfracture AT charlesathigpen assessingtheabilityofaninstrumentalvariablecausalforestalgorithmtopersonalizetreatmentevidenceusingobservationaldatathecaseofearlysurgeryforshoulderfracture AT michaelkissenberth assessingtheabilityofaninstrumentalvariablecausalforestalgorithmtopersonalizetreatmentevidenceusingobservationaldatathecaseofearlysurgeryforshoulderfracture |