Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study
OBJECTIVE: We examined how assessments of risk of bias of primary studies are carried out and incorporated into the statistical analysis and overall findings of a systematic review. DESIGN: A cross-sectional review. SAMPLE: We assessed 200 systematic reviews of randomised trials published between J...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
BMJ Publishing Group
2013
|
| _version_ | 1797069127428341760 |
|---|---|
| author | Hopewell, S Boutron, I Altman, D Ravaud, P |
| author_facet | Hopewell, S Boutron, I Altman, D Ravaud, P |
| author_sort | Hopewell, S |
| collection | OXFORD |
| description | OBJECTIVE: We examined how assessments of risk of bias of primary studies are carried out and incorporated into the statistical analysis and overall findings of a systematic review. DESIGN: A cross-sectional review. SAMPLE: We assessed 200 systematic reviews of randomised trials published between January and March 2012; Cochrane (n=100), non-Cochrane (Database of Reviews of Effects) (n=100). MAIN OUTCOMES: Our primary outcome was a descriptive analysis of how assessments of risk of bias are carried out, the methods used, and the extent to which such assessments were incorporated into the statistical analysis and overall review findings. RESULTS: While Cochrane reviews routinely reported the method of risk of bias assessment and presented their results either in text or table format, 20% of non-Cochrane reviews failed to report the method used and 39% did not present the assessment results. Where it was possible to evaluate the individual results of the risk of bias assessment (n=154), 75% (n=116/154) of reviews had ≥1 trial at high risk of bias; the median proportion of trials per review at high risk of bias was 50% (IQR 31% to 89%). Despite this, only 56% (n=65/116) incorporated the risk of bias assessment into the interpretation of the results in the abstract and 41% (n=47/116) (49%; n=40/81 Cochrane and 20%; n=7/35 non-Cochrane) incorporated the risk of bias assessment into the interpretation of the conclusions. Of the 83% (n=166/200) systematic reviews which included a meta-analysis, only 11% (n=19/166) incorporated the risk of bias assessment into the statistical analysis. CONCLUSIONS: Cochrane reviews were more likely than non-Cochrane reviews to report how risk of bias assessments of primary studies were carried out; however, both frequently failed to take such assessments into account in the statistical analysis and conclusions of the systematic review. |
| first_indexed | 2024-03-06T22:19:54Z |
| format | Journal article |
| id | oxford-uuid:54aa79a5-c29c-4a78-adda-079ed8b2784e |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T22:19:54Z |
| publishDate | 2013 |
| publisher | BMJ Publishing Group |
| record_format | dspace |
| spelling | oxford-uuid:54aa79a5-c29c-4a78-adda-079ed8b2784e2022-03-26T16:39:22ZIncorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional studyJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:54aa79a5-c29c-4a78-adda-079ed8b2784eEnglishSymplectic Elements at OxfordBMJ Publishing Group2013Hopewell, SBoutron, IAltman, DRavaud, P OBJECTIVE: We examined how assessments of risk of bias of primary studies are carried out and incorporated into the statistical analysis and overall findings of a systematic review. DESIGN: A cross-sectional review. SAMPLE: We assessed 200 systematic reviews of randomised trials published between January and March 2012; Cochrane (n=100), non-Cochrane (Database of Reviews of Effects) (n=100). MAIN OUTCOMES: Our primary outcome was a descriptive analysis of how assessments of risk of bias are carried out, the methods used, and the extent to which such assessments were incorporated into the statistical analysis and overall review findings. RESULTS: While Cochrane reviews routinely reported the method of risk of bias assessment and presented their results either in text or table format, 20% of non-Cochrane reviews failed to report the method used and 39% did not present the assessment results. Where it was possible to evaluate the individual results of the risk of bias assessment (n=154), 75% (n=116/154) of reviews had ≥1 trial at high risk of bias; the median proportion of trials per review at high risk of bias was 50% (IQR 31% to 89%). Despite this, only 56% (n=65/116) incorporated the risk of bias assessment into the interpretation of the results in the abstract and 41% (n=47/116) (49%; n=40/81 Cochrane and 20%; n=7/35 non-Cochrane) incorporated the risk of bias assessment into the interpretation of the conclusions. Of the 83% (n=166/200) systematic reviews which included a meta-analysis, only 11% (n=19/166) incorporated the risk of bias assessment into the statistical analysis. CONCLUSIONS: Cochrane reviews were more likely than non-Cochrane reviews to report how risk of bias assessments of primary studies were carried out; however, both frequently failed to take such assessments into account in the statistical analysis and conclusions of the systematic review. |
| spellingShingle | Hopewell, S Boutron, I Altman, D Ravaud, P Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study |
| title | Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study |
| title_full | Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study |
| title_fullStr | Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study |
| title_full_unstemmed | Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study |
| title_short | Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study |
| title_sort | incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials a cross sectional study |
| work_keys_str_mv | AT hopewells incorporationofassessmentsofriskofbiasofprimarystudiesinsystematicreviewsofrandomisedtrialsacrosssectionalstudy AT boutroni incorporationofassessmentsofriskofbiasofprimarystudiesinsystematicreviewsofrandomisedtrialsacrosssectionalstudy AT altmand incorporationofassessmentsofriskofbiasofprimarystudiesinsystematicreviewsofrandomisedtrialsacrosssectionalstudy AT ravaudp incorporationofassessmentsofriskofbiasofprimarystudiesinsystematicreviewsofrandomisedtrialsacrosssectionalstudy |