Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review
<p>Abstract</p> <p>Background</p> <p>Systematic reviewers seek to comprehensively search for relevant studies and summarize these to present the most valid estimate of intervention effectiveness. The more resources searched, the higher the yield, and thus time and costs...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2005-08-01
|
| Series: | BMC Medical Research Methodology |
| Online Access: | http://www.biomedcentral.com/1471-2288/5/24 |
| _version_ | 1811311419525693440 |
|---|---|
| author | Hartling Lisa Klassen Terry P Cramer Kristie Wiebe Natasha Crumley Ellen T |
| author_facet | Hartling Lisa Klassen Terry P Cramer Kristie Wiebe Natasha Crumley Ellen T |
| author_sort | Hartling Lisa |
| collection | DOAJ |
| description | <p>Abstract</p> <p>Background</p> <p>Systematic reviewers seek to comprehensively search for relevant studies and summarize these to present the most valid estimate of intervention effectiveness. The more resources searched, the higher the yield, and thus time and costs required to conduct a systematic review. While there is an abundance of evidence to suggest how extensive a search for randomized controlled trials (RCTs) should be, it is neither conclusive nor consistent. This systematic review was conducted in order to assess the value of different resources to identify trials for inclusion in systematic reviews.</p> <p>Methods</p> <p>Seven electronic databases, four journals and Cochrane Colloquia were searched. Key authors were contacted and references of relevant articles screened. Included studies compared two or more sources to find RCTs or controlled clinical trials (CCTs). A checklist was developed and applied to assess quality of reporting. Data were extracted by one reviewer and checked by a second. Medians and ranges for precision and recall were calculated; results were grouped by comparison. Meta-analysis was not performed due to large heterogeneity. Subgroup analyses were conducted for: search strategy (<it>Cochrane</it>, <it>Simple</it>, <it>Complex</it>, <it>Index</it>), expertise of the searcher (Cochrane, librarian, non-librarian), and study design (RCT and CCT).</p> <p>Results</p> <p>Sixty-four studies representing 13 electronic databases met inclusion criteria. The most common comparisons were MEDLINE vs. handsearching (n = 23), MEDLINE vs. MEDLINE+handsearching (n = 13), and MEDLINE vs. reference standard (n = 13). Quality was low, particularly for the reporting of study selection methodology. Overall, recall and precision varied substantially by comparison and ranged from 0 to 100% and 0 to 99%, respectively. The trial registries performed the best with median recall of 89% (range 84, 95) and median precision of 96.5% (96, 97), although these results are based on a small number of studies. Inadequate or inappropriate indexing was the reason most cited for missing studies. <it>Complex </it>and <it>Cochrane </it>search strategies (SS) performed better than <it>Simple </it>SS.</p> <p>Conclusion</p> <p>Multiple-source comprehensive searches are necessary to identify all RCTs for a systematic review, although indexing needs to be improved. Although trial registries demonstrated the highest recall and precision, the <it>Cochrane </it>SS or a <it>Complex </it>SS in consultation with a librarian are recommended. Continued efforts to develop CENTRAL should be supported.</p> |
| first_indexed | 2024-04-13T10:17:34Z |
| format | Article |
| id | doaj.art-8df222789ca6493284a5918817d8d0b8 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-13T10:17:34Z |
| publishDate | 2005-08-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-8df222789ca6493284a5918817d8d0b82022-12-22T02:50:39ZengBMCBMC Medical Research Methodology1471-22882005-08-01512410.1186/1471-2288-5-24Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic reviewHartling LisaKlassen Terry PCramer KristieWiebe NatashaCrumley Ellen T<p>Abstract</p> <p>Background</p> <p>Systematic reviewers seek to comprehensively search for relevant studies and summarize these to present the most valid estimate of intervention effectiveness. The more resources searched, the higher the yield, and thus time and costs required to conduct a systematic review. While there is an abundance of evidence to suggest how extensive a search for randomized controlled trials (RCTs) should be, it is neither conclusive nor consistent. This systematic review was conducted in order to assess the value of different resources to identify trials for inclusion in systematic reviews.</p> <p>Methods</p> <p>Seven electronic databases, four journals and Cochrane Colloquia were searched. Key authors were contacted and references of relevant articles screened. Included studies compared two or more sources to find RCTs or controlled clinical trials (CCTs). A checklist was developed and applied to assess quality of reporting. Data were extracted by one reviewer and checked by a second. Medians and ranges for precision and recall were calculated; results were grouped by comparison. Meta-analysis was not performed due to large heterogeneity. Subgroup analyses were conducted for: search strategy (<it>Cochrane</it>, <it>Simple</it>, <it>Complex</it>, <it>Index</it>), expertise of the searcher (Cochrane, librarian, non-librarian), and study design (RCT and CCT).</p> <p>Results</p> <p>Sixty-four studies representing 13 electronic databases met inclusion criteria. The most common comparisons were MEDLINE vs. handsearching (n = 23), MEDLINE vs. MEDLINE+handsearching (n = 13), and MEDLINE vs. reference standard (n = 13). Quality was low, particularly for the reporting of study selection methodology. Overall, recall and precision varied substantially by comparison and ranged from 0 to 100% and 0 to 99%, respectively. The trial registries performed the best with median recall of 89% (range 84, 95) and median precision of 96.5% (96, 97), although these results are based on a small number of studies. Inadequate or inappropriate indexing was the reason most cited for missing studies. <it>Complex </it>and <it>Cochrane </it>search strategies (SS) performed better than <it>Simple </it>SS.</p> <p>Conclusion</p> <p>Multiple-source comprehensive searches are necessary to identify all RCTs for a systematic review, although indexing needs to be improved. Although trial registries demonstrated the highest recall and precision, the <it>Cochrane </it>SS or a <it>Complex </it>SS in consultation with a librarian are recommended. Continued efforts to develop CENTRAL should be supported.</p>http://www.biomedcentral.com/1471-2288/5/24 |
| spellingShingle | Hartling Lisa Klassen Terry P Cramer Kristie Wiebe Natasha Crumley Ellen T Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review BMC Medical Research Methodology |
| title | Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review |
| title_full | Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review |
| title_fullStr | Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review |
| title_full_unstemmed | Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review |
| title_short | Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review |
| title_sort | which resources should be used to identify rct ccts for systematic reviews a systematic review |
| url | http://www.biomedcentral.com/1471-2288/5/24 |
| work_keys_str_mv | AT hartlinglisa whichresourcesshouldbeusedtoidentifyrctcctsforsystematicreviewsasystematicreview AT klassenterryp whichresourcesshouldbeusedtoidentifyrctcctsforsystematicreviewsasystematicreview AT cramerkristie whichresourcesshouldbeusedtoidentifyrctcctsforsystematicreviewsasystematicreview AT wiebenatasha whichresourcesshouldbeusedtoidentifyrctcctsforsystematicreviewsasystematicreview AT crumleyellent whichresourcesshouldbeusedtoidentifyrctcctsforsystematicreviewsasystematicreview |