Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook
Abstract Background Assessing the risk of bias (RoB) in included studies is one of the key methodological aspects of systematic reviews. Cochrane systematic reviews appraise RoB of randomised controlled trials (RCTs) with the Cochrane RoB tool. Detailed instructions for using the Cochrane RoB tool a...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2019-08-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12874-019-0804-y |
| _version_ | 1818964266015784960 |
|---|---|
| author | Ognjen Barcot Matija Boric Tina Poklepovic Pericic Marija Cavar Svjetlana Dosenovic Ivana Vuka Livia Puljak |
| author_facet | Ognjen Barcot Matija Boric Tina Poklepovic Pericic Marija Cavar Svjetlana Dosenovic Ivana Vuka Livia Puljak |
| author_sort | Ognjen Barcot |
| collection | DOAJ |
| description | Abstract Background Assessing the risk of bias (RoB) in included studies is one of the key methodological aspects of systematic reviews. Cochrane systematic reviews appraise RoB of randomised controlled trials (RCTs) with the Cochrane RoB tool. Detailed instructions for using the Cochrane RoB tool are provided in the Cochrane Handbook for Systematic Reviews of Interventions (The Cochrane Handbook). The purpose of this study was to analyse whether Cochrane authors use adequate judgments about the RoB for random sequence generation of RCTs included in Cochrane reviews. Methods We extracted authors’ judgments (high, low or unclear RoB) and supports for judgments (comments accompanying judgments which explain the rationale for a judgment) for random sequence generation of included RCTs from RoB tables of Cochrane reviews using automated data scraping. We categorised all supporting comments, analysed the number and type of various supporting comments and assessed adequacy of RoB judgment for randomisation in line with recommendations from the Cochrane Handbook. Results We analysed 10,103 RCTs that were included in 704 Cochrane reviews. For 5,706 RCTs, randomisation was not described, but for the remaining RCTs, it was indicated that randomisation was performed using computer/software/internet (N = 2,850), random number table (N = 883), mechanical method (N = 359) or it was incomplete/inappropriate (N = 305). Overall, 1,220/10,103 trials (12%) did not have a RoB judgment in line with Cochrane Handbook guidance about randomisation. The highest proportion of misjudgements was found for trials with high RoB (28%), followed by those with low (20%) or unclear (3%). Therefore, one in eight judgments for the analysed domain in Cochrane reviews was not in line with Cochrane Handbook, and one in four if the judgment was "high risk". Conclusion Authors of Cochrane reviews often make judgments about the RoB related to random sequence generation that are not in line with instructions given in the Cochrane Handbook, which compromises the reliability of the systematic reviews. Our results can help authors of both Cochrane and non-Cochrane reviews which use Cochrane RoB tool to avoid making common mistakes when assessing RoB in included trials. |
| first_indexed | 2024-12-20T12:58:22Z |
| format | Article |
| id | doaj.art-5858e94013a94a01a85f20ebd2b6d60f |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-20T12:58:22Z |
| publishDate | 2019-08-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-5858e94013a94a01a85f20ebd2b6d60f2022-12-21T19:40:00ZengBMCBMC Medical Research Methodology1471-22882019-08-0119111010.1186/s12874-019-0804-yRisk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane HandbookOgnjen Barcot0Matija Boric1Tina Poklepovic Pericic2Marija Cavar3Svjetlana Dosenovic4Ivana Vuka5Livia Puljak6Department of Surgery, University Hospital SplitDepartment of Surgery, University Hospital SplitDepartment for Research in Biomedicine and Health, University of Split, School of MedicineDepartment of Radiology, University Hospital SplitDepartment of Anesthesiology and Intensive Care, University Hospital SplitDepartment for Research in Biomedicine and Health, University of Split, School of MedicineCenter for Evidence-Based Medicine and Health Care, Catholic University of CroatiaAbstract Background Assessing the risk of bias (RoB) in included studies is one of the key methodological aspects of systematic reviews. Cochrane systematic reviews appraise RoB of randomised controlled trials (RCTs) with the Cochrane RoB tool. Detailed instructions for using the Cochrane RoB tool are provided in the Cochrane Handbook for Systematic Reviews of Interventions (The Cochrane Handbook). The purpose of this study was to analyse whether Cochrane authors use adequate judgments about the RoB for random sequence generation of RCTs included in Cochrane reviews. Methods We extracted authors’ judgments (high, low or unclear RoB) and supports for judgments (comments accompanying judgments which explain the rationale for a judgment) for random sequence generation of included RCTs from RoB tables of Cochrane reviews using automated data scraping. We categorised all supporting comments, analysed the number and type of various supporting comments and assessed adequacy of RoB judgment for randomisation in line with recommendations from the Cochrane Handbook. Results We analysed 10,103 RCTs that were included in 704 Cochrane reviews. For 5,706 RCTs, randomisation was not described, but for the remaining RCTs, it was indicated that randomisation was performed using computer/software/internet (N = 2,850), random number table (N = 883), mechanical method (N = 359) or it was incomplete/inappropriate (N = 305). Overall, 1,220/10,103 trials (12%) did not have a RoB judgment in line with Cochrane Handbook guidance about randomisation. The highest proportion of misjudgements was found for trials with high RoB (28%), followed by those with low (20%) or unclear (3%). Therefore, one in eight judgments for the analysed domain in Cochrane reviews was not in line with Cochrane Handbook, and one in four if the judgment was "high risk". Conclusion Authors of Cochrane reviews often make judgments about the RoB related to random sequence generation that are not in line with instructions given in the Cochrane Handbook, which compromises the reliability of the systematic reviews. Our results can help authors of both Cochrane and non-Cochrane reviews which use Cochrane RoB tool to avoid making common mistakes when assessing RoB in included trials.http://link.springer.com/article/10.1186/s12874-019-0804-yRisk of biasCochraneSystematic reviewsRandomisationSequence generationSelection bias |
| spellingShingle | Ognjen Barcot Matija Boric Tina Poklepovic Pericic Marija Cavar Svjetlana Dosenovic Ivana Vuka Livia Puljak Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook BMC Medical Research Methodology Risk of bias Cochrane Systematic reviews Randomisation Sequence generation Selection bias |
| title | Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook |
| title_full | Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook |
| title_fullStr | Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook |
| title_full_unstemmed | Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook |
| title_short | Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook |
| title_sort | risk of bias judgments for random sequence generation in cochrane systematic reviews were frequently not in line with cochrane handbook |
| topic | Risk of bias Cochrane Systematic reviews Randomisation Sequence generation Selection bias |
| url | http://link.springer.com/article/10.1186/s12874-019-0804-y |
| work_keys_str_mv | AT ognjenbarcot riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook AT matijaboric riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook AT tinapoklepovicpericic riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook AT marijacavar riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook AT svjetlanadosenovic riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook AT ivanavuka riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook AT liviapuljak riskofbiasjudgmentsforrandomsequencegenerationincochranesystematicreviewswerefrequentlynotinlinewithcochranehandbook |