Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study
Objective To examine how poor reporting and inadequate methods for key methodological features in randomised controlled trials (RCTs) have changed over the past three decades. Design Mapping of trials included in Cochrane reviews. Data sources Data from RCTs included in all Cochrane reviews publis...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Journal article |
| Published: |
BMJ Publishing Group
2017
|
| _version_ | 1826296749426212864 |
|---|---|
| author | Dechartres, A Trinquart, L Atal, I Moher, D Dickersin, K Boutron, I Perrodeau, E Altman, D Ravaud, P |
| author_facet | Dechartres, A Trinquart, L Atal, I Moher, D Dickersin, K Boutron, I Perrodeau, E Altman, D Ravaud, P |
| author_sort | Dechartres, A |
| collection | OXFORD |
| description | Objective To examine how poor reporting and inadequate methods for key methodological features in randomised controlled trials (RCTs) have changed over the past three decades. Design Mapping of trials included in Cochrane reviews. Data sources Data from RCTs included in all Cochrane reviews published between March 2011 and September 2014 reporting an evaluation of the Cochrane risk of bias items: sequence generation, allocation concealment, blinding, and incomplete outcome data. Data extraction For each RCT, we extracted consensus on risk of bias made by the review authors and identified the primary reference to extract publication year and journal. We matched journal names with Journal Citation Reports to get 2014 impact factors. Main outcomes measures We considered the proportions of trials rated by review authors at unclear and high risk of bias as surrogates for poor reporting and inadequate methods, respectively. Results We analysed 20 920 RCTs (from 2001 reviews) published in 3136 journals. The proportion of trials with unclear risk of bias was 48.7% for sequence generation and 57.5% for allocation concealment; the proportion of those with high risk of bias was 4.0% and 7.2%, respectively. For blinding and incomplete outcome data, 30.6% and 24.7% of trials were at unclear risk and 33.1% and 17.1% were at high risk, respectively. Higher journal impact factor was associated with a lower proportion of trials at unclear or high risk of bias. The proportion of trials at unclear risk of bias decreased over time, especially for sequence generation, which fell from 69.1% in 1986-1990 to 31.2% in 2011-14 and for allocation concealment (70.1% to 44.6%). After excluding trials at unclear risk of bias, use of inadequate methods also decreased over time: from 14.8% to 4.6% for sequence generation and from 32.7% to 11.6% for allocation concealment. Conclusions Poor reporting and inadequate methods have decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals. |
| first_indexed | 2024-03-07T04:21:08Z |
| format | Journal article |
| id | oxford-uuid:cb0f8466-a761-4bd4-9172-9b5f6c6276b8 |
| institution | University of Oxford |
| last_indexed | 2024-03-07T04:21:08Z |
| publishDate | 2017 |
| publisher | BMJ Publishing Group |
| record_format | dspace |
| spelling | oxford-uuid:cb0f8466-a761-4bd4-9172-9b5f6c6276b82022-03-27T07:12:08ZEvolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research studyJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:cb0f8466-a761-4bd4-9172-9b5f6c6276b8Symplectic Elements at OxfordBMJ Publishing Group2017Dechartres, ATrinquart, LAtal, IMoher, DDickersin, KBoutron, IPerrodeau, EAltman, DRavaud, PObjective To examine how poor reporting and inadequate methods for key methodological features in randomised controlled trials (RCTs) have changed over the past three decades. Design Mapping of trials included in Cochrane reviews. Data sources Data from RCTs included in all Cochrane reviews published between March 2011 and September 2014 reporting an evaluation of the Cochrane risk of bias items: sequence generation, allocation concealment, blinding, and incomplete outcome data. Data extraction For each RCT, we extracted consensus on risk of bias made by the review authors and identified the primary reference to extract publication year and journal. We matched journal names with Journal Citation Reports to get 2014 impact factors. Main outcomes measures We considered the proportions of trials rated by review authors at unclear and high risk of bias as surrogates for poor reporting and inadequate methods, respectively. Results We analysed 20 920 RCTs (from 2001 reviews) published in 3136 journals. The proportion of trials with unclear risk of bias was 48.7% for sequence generation and 57.5% for allocation concealment; the proportion of those with high risk of bias was 4.0% and 7.2%, respectively. For blinding and incomplete outcome data, 30.6% and 24.7% of trials were at unclear risk and 33.1% and 17.1% were at high risk, respectively. Higher journal impact factor was associated with a lower proportion of trials at unclear or high risk of bias. The proportion of trials at unclear risk of bias decreased over time, especially for sequence generation, which fell from 69.1% in 1986-1990 to 31.2% in 2011-14 and for allocation concealment (70.1% to 44.6%). After excluding trials at unclear risk of bias, use of inadequate methods also decreased over time: from 14.8% to 4.6% for sequence generation and from 32.7% to 11.6% for allocation concealment. Conclusions Poor reporting and inadequate methods have decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals. |
| spellingShingle | Dechartres, A Trinquart, L Atal, I Moher, D Dickersin, K Boutron, I Perrodeau, E Altman, D Ravaud, P Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study |
| title | Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study |
| title_full | Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study |
| title_fullStr | Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study |
| title_full_unstemmed | Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study |
| title_short | Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study |
| title_sort | evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in cochrane reviews research on research study |
| work_keys_str_mv | AT dechartresa evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT trinquartl evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT atali evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT moherd evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT dickersink evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT boutroni evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT perrodeaue evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT altmand evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy AT ravaudp evolutionofpoorreportingandinadequatemethodsovertimein20920randomisedcontrolledtrialsincludedincochranereviewsresearchonresearchstudy |