Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study
Abstract Background In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clu...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2018-10-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12874-018-0559-x |
| _version_ | 1818451222866165760 |
|---|---|
| author | Jane Candlish M. Dawn Teare Munyaradzi Dimairo Laura Flight Laura Mandefield Stephen J. Walters |
| author_facet | Jane Candlish M. Dawn Teare Munyaradzi Dimairo Laura Flight Laura Mandefield Stephen J. Walters |
| author_sort | Jane Candlish |
| collection | DOAJ |
| description | Abstract Background In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis. Methods We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms. Results All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms. Conclusions In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3–6), small cluster sizes (5–10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model. |
| first_indexed | 2024-12-14T21:03:46Z |
| format | Article |
| id | doaj.art-e995d3f20f51476d931319c2d875e825 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-14T21:03:46Z |
| publishDate | 2018-10-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-e995d3f20f51476d931319c2d875e8252022-12-21T22:47:31ZengBMCBMC Medical Research Methodology1471-22882018-10-0118111710.1186/s12874-018-0559-xAppropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation studyJane Candlish0M. Dawn Teare1Munyaradzi Dimairo2Laura Flight3Laura Mandefield4Stephen J. Walters5School of Health and Related Research (ScHARR), University of SheffieldSchool of Health and Related Research (ScHARR), University of SheffieldSchool of Health and Related Research (ScHARR), University of SheffieldSchool of Health and Related Research (ScHARR), University of SheffieldSchool of Health and Related Research (ScHARR), University of SheffieldSchool of Health and Related Research (ScHARR), University of SheffieldAbstract Background In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis. Methods We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms. Results All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms. Conclusions In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3–6), small cluster sizes (5–10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model.http://link.springer.com/article/10.1186/s12874-018-0559-xClusteringRandomised controlled trialPartially nestedPartially clusteredTherapist effectsIndividually randomised group treatment |
| spellingShingle | Jane Candlish M. Dawn Teare Munyaradzi Dimairo Laura Flight Laura Mandefield Stephen J. Walters Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study BMC Medical Research Methodology Clustering Randomised controlled trial Partially nested Partially clustered Therapist effects Individually randomised group treatment |
| title | Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study |
| title_full | Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study |
| title_fullStr | Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study |
| title_full_unstemmed | Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study |
| title_short | Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study |
| title_sort | appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes a simulation study |
| topic | Clustering Randomised controlled trial Partially nested Partially clustered Therapist effects Individually randomised group treatment |
| url | http://link.springer.com/article/10.1186/s12874-018-0559-x |
| work_keys_str_mv | AT janecandlish appropriatestatisticalmethodsforanalysingpartiallynestedrandomisedcontrolledtrialswithcontinuousoutcomesasimulationstudy AT mdawnteare appropriatestatisticalmethodsforanalysingpartiallynestedrandomisedcontrolledtrialswithcontinuousoutcomesasimulationstudy AT munyaradzidimairo appropriatestatisticalmethodsforanalysingpartiallynestedrandomisedcontrolledtrialswithcontinuousoutcomesasimulationstudy AT lauraflight appropriatestatisticalmethodsforanalysingpartiallynestedrandomisedcontrolledtrialswithcontinuousoutcomesasimulationstudy AT lauramandefield appropriatestatisticalmethodsforanalysingpartiallynestedrandomisedcontrolledtrialswithcontinuousoutcomesasimulationstudy AT stephenjwalters appropriatestatisticalmethodsforanalysingpartiallynestedrandomisedcontrolledtrialswithcontinuousoutcomesasimulationstudy |