A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials
Abstract Introduction Non-compliance is a common challenge for researchers and may reduce the power of an intention-to-treat analysis. Whilst a per protocol approach attempts to deal with this issue, it can result in biased estimates. Several methods to resolve this issue have been identified in pre...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-12-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-023-02126-w |
| _version_ | 1797388277516337152 |
|---|---|
| author | Lucy Abell Francesca Maher Angus C Jennings Laura J Gray |
| author_facet | Lucy Abell Francesca Maher Angus C Jennings Laura J Gray |
| author_sort | Lucy Abell |
| collection | DOAJ |
| description | Abstract Introduction Non-compliance is a common challenge for researchers and may reduce the power of an intention-to-treat analysis. Whilst a per protocol approach attempts to deal with this issue, it can result in biased estimates. Several methods to resolve this issue have been identified in previous reviews, but there is limited evidence supporting their use. This review aimed to identify simulation studies which compare such methods, assess the extent to which certain methods have been investigated and determine their performance under various scenarios. Methods A systematic search of several electronic databases including MEDLINE and Scopus was carried out from conception to 30th November 2022. Included papers were published in a peer-reviewed journal, readily available in the English language and focused on comparing relevant methods in a superiority randomised controlled trial under a simulation study. Articles were screened using these criteria and a predetermined extraction form used to identify relevant information. A quality assessment appraised the risk of bias in individual studies. Extracted data was synthesised using tables, figures and a narrative summary. Both screening and data extraction were performed by two independent reviewers with disagreements resolved by consensus. Results Of 2325 papers identified, 267 full texts were screened and 17 studies finally included. Twelve methods were identified across papers. Instrumental variable methods were commonly considered, but many authors found them to be biased in some settings. Non-compliance was generally assumed to be all-or-nothing and only occurring in the intervention group, although some methods considered it as time-varying. Simulation studies commonly varied the level and type of non-compliance and factors such as effect size and strength of confounding. The quality of papers was generally good, although some lacked detail and justification. Therefore, their conclusions were deemed to be less reliable. Conclusions It is common for papers to consider instrumental variable methods but more studies are needed that consider G-methods and compare a wide range of methods in realistic scenarios. It is difficult to make conclusions about the best method to deal with non-compliance due to a limited body of evidence and the difficulty in combining results from independent simulation studies. PROSPERO registration number CRD42022370910. |
| first_indexed | 2024-03-08T22:37:30Z |
| format | Article |
| id | doaj.art-8796d943ad1b4054a3918033f412e19a |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-03-08T22:37:30Z |
| publishDate | 2023-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-8796d943ad1b4054a3918033f412e19a2023-12-17T12:21:26ZengBMCBMC Medical Research Methodology1471-22882023-12-0123111910.1186/s12874-023-02126-wA systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trialsLucy Abell0Francesca Maher1Angus C Jennings2Laura J Gray3Department of Population Health Sciences, University of LeicesterDepartment of Population Health Sciences, University of LeicesterDepartment of Population Health Sciences, University of LeicesterDepartment of Population Health Sciences, University of LeicesterAbstract Introduction Non-compliance is a common challenge for researchers and may reduce the power of an intention-to-treat analysis. Whilst a per protocol approach attempts to deal with this issue, it can result in biased estimates. Several methods to resolve this issue have been identified in previous reviews, but there is limited evidence supporting their use. This review aimed to identify simulation studies which compare such methods, assess the extent to which certain methods have been investigated and determine their performance under various scenarios. Methods A systematic search of several electronic databases including MEDLINE and Scopus was carried out from conception to 30th November 2022. Included papers were published in a peer-reviewed journal, readily available in the English language and focused on comparing relevant methods in a superiority randomised controlled trial under a simulation study. Articles were screened using these criteria and a predetermined extraction form used to identify relevant information. A quality assessment appraised the risk of bias in individual studies. Extracted data was synthesised using tables, figures and a narrative summary. Both screening and data extraction were performed by two independent reviewers with disagreements resolved by consensus. Results Of 2325 papers identified, 267 full texts were screened and 17 studies finally included. Twelve methods were identified across papers. Instrumental variable methods were commonly considered, but many authors found them to be biased in some settings. Non-compliance was generally assumed to be all-or-nothing and only occurring in the intervention group, although some methods considered it as time-varying. Simulation studies commonly varied the level and type of non-compliance and factors such as effect size and strength of confounding. The quality of papers was generally good, although some lacked detail and justification. Therefore, their conclusions were deemed to be less reliable. Conclusions It is common for papers to consider instrumental variable methods but more studies are needed that consider G-methods and compare a wide range of methods in realistic scenarios. It is difficult to make conclusions about the best method to deal with non-compliance due to a limited body of evidence and the difficulty in combining results from independent simulation studies. PROSPERO registration number CRD42022370910.https://doi.org/10.1186/s12874-023-02126-wNon-complianceSimulation studiesStatistical methodsRandomised controlled trials |
| spellingShingle | Lucy Abell Francesca Maher Angus C Jennings Laura J Gray A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials BMC Medical Research Methodology Non-compliance Simulation studies Statistical methods Randomised controlled trials |
| title | A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials |
| title_full | A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials |
| title_fullStr | A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials |
| title_full_unstemmed | A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials |
| title_short | A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials |
| title_sort | systematic review of simulation studies which compare existing statistical methods to account for non compliance in randomised controlled trials |
| topic | Non-compliance Simulation studies Statistical methods Randomised controlled trials |
| url | https://doi.org/10.1186/s12874-023-02126-w |
| work_keys_str_mv | AT lucyabell asystematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT francescamaher asystematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT anguscjennings asystematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT laurajgray asystematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT lucyabell systematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT francescamaher systematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT anguscjennings systematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials AT laurajgray systematicreviewofsimulationstudieswhichcompareexistingstatisticalmethodstoaccountfornoncomplianceinrandomisedcontrolledtrials |