Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters
Abstract Background Stepped wedge trials are an appealing and potentially powerful cluster randomized trial design. However, they are frequently implemented with a small number of clusters. Standard analysis methods for these trials such as a linear mixed model with estimation via maximum likelihood...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-04-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-022-01550-8 |
| _version_ | 1828405471673843712 |
|---|---|
| author | Kelsey L. Grantham Jessica Kasza Stephane Heritier John B. Carlin Andrew B. Forbes |
| author_facet | Kelsey L. Grantham Jessica Kasza Stephane Heritier John B. Carlin Andrew B. Forbes |
| author_sort | Kelsey L. Grantham |
| collection | DOAJ |
| description | Abstract Background Stepped wedge trials are an appealing and potentially powerful cluster randomized trial design. However, they are frequently implemented with a small number of clusters. Standard analysis methods for these trials such as a linear mixed model with estimation via maximum likelihood or restricted maximum likelihood (REML) rely on asymptotic properties and have been shown to yield inflated type I error when applied to studies with a small number of clusters. Small-sample methods such as the Kenward-Roger approximation in combination with REML can potentially improve estimation of the fixed effects such as the treatment effect. A Bayesian approach may also be promising for such multilevel models but has not yet seen much application in cluster randomized trials. Methods We conducted a simulation study comparing the performance of REML with and without a Kenward-Roger approximation to a Bayesian approach using weakly informative prior distributions on the intracluster correlation parameters. We considered a continuous outcome and a range of stepped wedge trial configurations with between 4 and 40 clusters. To assess method performance we calculated bias and mean squared error for the treatment effect and correlation parameters and the coverage of 95% confidence/credible intervals and relative percent error in model-based standard error for the treatment effect. Results Both REML with a Kenward-Roger standard error and degrees of freedom correction and the Bayesian method performed similarly well for the estimation of the treatment effect, while intracluster correlation parameter estimates obtained via the Bayesian method were less variable than REML estimates with different relative levels of bias. Conclusions The use of REML with a Kenward-Roger approximation may be sufficient for the analysis of stepped wedge cluster randomized trials with a small number of clusters. However, a Bayesian approach with weakly informative prior distributions on the intracluster correlation parameters offers a viable alternative, particularly when there is interest in the probability-based inferences permitted within this paradigm. |
| first_indexed | 2024-12-10T10:53:54Z |
| format | Article |
| id | doaj.art-9aafa8b91ba44d3abcb9969d7c5019dd |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-10T10:53:54Z |
| publishDate | 2022-04-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-9aafa8b91ba44d3abcb9969d7c5019dd2022-12-22T01:51:56ZengBMCBMC Medical Research Methodology1471-22882022-04-0122111810.1186/s12874-022-01550-8Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clustersKelsey L. Grantham0Jessica Kasza1Stephane Heritier2John B. Carlin3Andrew B. Forbes4School of Public Health and Preventive Medicine, Monash UniversitySchool of Public Health and Preventive Medicine, Monash UniversitySchool of Public Health and Preventive Medicine, Monash UniversityClinical Epidemiology and Biostatistics Unit, Murdoch Children’s Research InstituteSchool of Public Health and Preventive Medicine, Monash UniversityAbstract Background Stepped wedge trials are an appealing and potentially powerful cluster randomized trial design. However, they are frequently implemented with a small number of clusters. Standard analysis methods for these trials such as a linear mixed model with estimation via maximum likelihood or restricted maximum likelihood (REML) rely on asymptotic properties and have been shown to yield inflated type I error when applied to studies with a small number of clusters. Small-sample methods such as the Kenward-Roger approximation in combination with REML can potentially improve estimation of the fixed effects such as the treatment effect. A Bayesian approach may also be promising for such multilevel models but has not yet seen much application in cluster randomized trials. Methods We conducted a simulation study comparing the performance of REML with and without a Kenward-Roger approximation to a Bayesian approach using weakly informative prior distributions on the intracluster correlation parameters. We considered a continuous outcome and a range of stepped wedge trial configurations with between 4 and 40 clusters. To assess method performance we calculated bias and mean squared error for the treatment effect and correlation parameters and the coverage of 95% confidence/credible intervals and relative percent error in model-based standard error for the treatment effect. Results Both REML with a Kenward-Roger standard error and degrees of freedom correction and the Bayesian method performed similarly well for the estimation of the treatment effect, while intracluster correlation parameter estimates obtained via the Bayesian method were less variable than REML estimates with different relative levels of bias. Conclusions The use of REML with a Kenward-Roger approximation may be sufficient for the analysis of stepped wedge cluster randomized trials with a small number of clusters. However, a Bayesian approach with weakly informative prior distributions on the intracluster correlation parameters offers a viable alternative, particularly when there is interest in the probability-based inferences permitted within this paradigm.https://doi.org/10.1186/s12874-022-01550-8Bayesian inferenceCluster randomized trialIntracluster correlationRestricted maximum likelihoodSimulation studyStepped wedge |
| spellingShingle | Kelsey L. Grantham Jessica Kasza Stephane Heritier John B. Carlin Andrew B. Forbes Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters BMC Medical Research Methodology Bayesian inference Cluster randomized trial Intracluster correlation Restricted maximum likelihood Simulation study Stepped wedge |
| title | Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters |
| title_full | Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters |
| title_fullStr | Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters |
| title_full_unstemmed | Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters |
| title_short | Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters |
| title_sort | evaluating the performance of bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters |
| topic | Bayesian inference Cluster randomized trial Intracluster correlation Restricted maximum likelihood Simulation study Stepped wedge |
| url | https://doi.org/10.1186/s12874-022-01550-8 |
| work_keys_str_mv | AT kelseylgrantham evaluatingtheperformanceofbayesianandrestrictedmaximumlikelihoodestimationforsteppedwedgeclusterrandomizedtrialswithasmallnumberofclusters AT jessicakasza evaluatingtheperformanceofbayesianandrestrictedmaximumlikelihoodestimationforsteppedwedgeclusterrandomizedtrialswithasmallnumberofclusters AT stephaneheritier evaluatingtheperformanceofbayesianandrestrictedmaximumlikelihoodestimationforsteppedwedgeclusterrandomizedtrialswithasmallnumberofclusters AT johnbcarlin evaluatingtheperformanceofbayesianandrestrictedmaximumlikelihoodestimationforsteppedwedgeclusterrandomizedtrialswithasmallnumberofclusters AT andrewbforbes evaluatingtheperformanceofbayesianandrestrictedmaximumlikelihoodestimationforsteppedwedgeclusterrandomizedtrialswithasmallnumberofclusters |