Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring
Abstract Background Numerous clinical trials have been initiated to find effective treatments for COVID-19. These trials have often been initiated in regions where the pandemic has already peaked. Consequently, achieving full enrollment in a single trial might require additional COVID-19 surges in t...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-01-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-022-01813-4 |
| _version_ | 1828051630166114304 |
|---|---|
| author | Danni Wu Keith S. Goldfeld Eva Petkova |
| author_facet | Danni Wu Keith S. Goldfeld Eva Petkova |
| author_sort | Danni Wu |
| collection | DOAJ |
| description | Abstract Background Numerous clinical trials have been initiated to find effective treatments for COVID-19. These trials have often been initiated in regions where the pandemic has already peaked. Consequently, achieving full enrollment in a single trial might require additional COVID-19 surges in the same location over several years. This has inspired us to pool individual patient data (IPD) from ongoing, paused, prematurely-terminated, or completed randomized controlled trials (RCTs) in real-time, to find an effective treatment as quickly as possible in light of the pandemic crisis. However, pooling across trials introduces enormous uncertainties in study design (e.g., the number of RCTs and sample sizes might be unknown in advance). We sought to develop a versatile treatment efficacy assessment model that accounts for these uncertainties while allowing for continuous monitoring throughout the study using Bayesian monitoring techniques. Methods We provide a detailed look at the challenges and solutions for model development, describing the process that used extensive simulations to enable us to finalize the analysis plan. This includes establishing prior distribution assumptions, assessing and improving model convergence under different study composition scenarios, and assessing whether we can extend the model to accommodate multi-site RCTs and evaluate heterogeneous treatment effects. In addition, we recognized that we would need to assess our model for goodness-of-fit, so we explored an approach that used posterior predictive checking. Lastly, given the urgency of the research in the context of evolving pandemic, we were committed to frequent monitoring of the data to assess efficacy, and we set Bayesian monitoring rules calibrated for type 1 error rate and power. Results The primary outcome is an 11-point ordinal scale. We present the operating characteristics of the proposed cumulative proportional odds model for estimating treatment effectiveness. The model can estimate the treatment’s effect under enormous uncertainties in study design. We investigate to what degree the proportional odds assumption has to be violated to render the model inaccurate. We demonstrate the flexibility of a Bayesian monitoring approach by performing frequent interim analyses without increasing the probability of erroneous conclusions. Conclusion This paper describes a translatable framework using simulation to support the design of prospective IPD meta-analyses. |
| first_indexed | 2024-04-10T19:41:36Z |
| format | Article |
| id | doaj.art-10812d65e65e4c389e1f4bb251706ddc |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-10T19:41:36Z |
| publishDate | 2023-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-10812d65e65e4c389e1f4bb251706ddc2023-01-29T12:15:46ZengBMCBMC Medical Research Methodology1471-22882023-01-0123111510.1186/s12874-022-01813-4Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoringDanni Wu0Keith S. Goldfeld1Eva Petkova2Department of Population Health, New York University Grossman School of MedicineDepartment of Population Health, New York University Grossman School of MedicineDepartment of Population Health, New York University Grossman School of MedicineAbstract Background Numerous clinical trials have been initiated to find effective treatments for COVID-19. These trials have often been initiated in regions where the pandemic has already peaked. Consequently, achieving full enrollment in a single trial might require additional COVID-19 surges in the same location over several years. This has inspired us to pool individual patient data (IPD) from ongoing, paused, prematurely-terminated, or completed randomized controlled trials (RCTs) in real-time, to find an effective treatment as quickly as possible in light of the pandemic crisis. However, pooling across trials introduces enormous uncertainties in study design (e.g., the number of RCTs and sample sizes might be unknown in advance). We sought to develop a versatile treatment efficacy assessment model that accounts for these uncertainties while allowing for continuous monitoring throughout the study using Bayesian monitoring techniques. Methods We provide a detailed look at the challenges and solutions for model development, describing the process that used extensive simulations to enable us to finalize the analysis plan. This includes establishing prior distribution assumptions, assessing and improving model convergence under different study composition scenarios, and assessing whether we can extend the model to accommodate multi-site RCTs and evaluate heterogeneous treatment effects. In addition, we recognized that we would need to assess our model for goodness-of-fit, so we explored an approach that used posterior predictive checking. Lastly, given the urgency of the research in the context of evolving pandemic, we were committed to frequent monitoring of the data to assess efficacy, and we set Bayesian monitoring rules calibrated for type 1 error rate and power. Results The primary outcome is an 11-point ordinal scale. We present the operating characteristics of the proposed cumulative proportional odds model for estimating treatment effectiveness. The model can estimate the treatment’s effect under enormous uncertainties in study design. We investigate to what degree the proportional odds assumption has to be violated to render the model inaccurate. We demonstrate the flexibility of a Bayesian monitoring approach by performing frequent interim analyses without increasing the probability of erroneous conclusions. Conclusion This paper describes a translatable framework using simulation to support the design of prospective IPD meta-analyses.https://doi.org/10.1186/s12874-022-01813-4Bayesian hierarchical modelsBayesian adaptive trial designBayesian simulationInternational consortium for data sharingProspective individual patient data meta-analysisCOVID-19 |
| spellingShingle | Danni Wu Keith S. Goldfeld Eva Petkova Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring BMC Medical Research Methodology Bayesian hierarchical models Bayesian adaptive trial design Bayesian simulation International consortium for data sharing Prospective individual patient data meta-analysis COVID-19 |
| title | Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring |
| title_full | Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring |
| title_fullStr | Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring |
| title_full_unstemmed | Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring |
| title_short | Developing a Bayesian hierarchical model for a prospective individual patient data meta-analysis with continuous monitoring |
| title_sort | developing a bayesian hierarchical model for a prospective individual patient data meta analysis with continuous monitoring |
| topic | Bayesian hierarchical models Bayesian adaptive trial design Bayesian simulation International consortium for data sharing Prospective individual patient data meta-analysis COVID-19 |
| url | https://doi.org/10.1186/s12874-022-01813-4 |
| work_keys_str_mv | AT danniwu developingabayesianhierarchicalmodelforaprospectiveindividualpatientdatametaanalysiswithcontinuousmonitoring AT keithsgoldfeld developingabayesianhierarchicalmodelforaprospectiveindividualpatientdatametaanalysiswithcontinuousmonitoring AT evapetkova developingabayesianhierarchicalmodelforaprospectiveindividualpatientdatametaanalysiswithcontinuousmonitoring |