An alternative to the hand searching gold standard: validating methodological search filters using relative recall
<p>Abstract</p> <p>Background</p> <p>Search filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard" – a reference standard against which to establish the performance of the filt...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2006-07-01
|
| Series: | BMC Medical Research Methodology |
| Online Access: | http://www.biomedcentral.com/1471-2288/6/33 |
| _version_ | 1811248902231293952 |
|---|---|
| author | Barrowman Nicholas J Morrison Andra Zhang Li Sampson Margaret Clifford Tammy J Platt Robert W Klassen Terry P Moher David |
| author_facet | Barrowman Nicholas J Morrison Andra Zhang Li Sampson Margaret Clifford Tammy J Platt Robert W Klassen Terry P Moher David |
| author_sort | Barrowman Nicholas J |
| collection | DOAJ |
| description | <p>Abstract</p> <p>Background</p> <p>Search filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard" – a reference standard against which to establish the performance of the filter. We demonstrate the feasibility of using relative recall of included studies from multiple systematic reviews to validate methodological search filters as an alternative to validation against a gold standard formed through hand searching.</p> <p>Methods</p> <p>We identified 105 Cochrane reviews that used the Highly Sensitive Search Strategy (HSSS), included randomized or quasi-randomized controlled trials, and reported their included studies. We measured the ability of two published and one novel variant of the HSSS to retrieve the MEDLINE-index studies included in these reviews.</p> <p>Results</p> <p>The systematic reviews were comprehensive in their searches. 72% of included primary studies were indexed in MEDLINE. Relative recall of the three strategies ranged from .98 to .91 across all reviews and more comprehensive strategies showed higher recall.</p> <p>Conclusion</p> <p>An approach using relative recall instead of a hand searching gold standard proved feasible and produced recall figures that were congruent with previously published figures for the HSSS. This technique would permit validation of a methodological filter using a collection of approximately 100 studies of the chosen design drawn from the included studies of multiple systematic reviews that used comprehensive search strategies.</p> |
| first_indexed | 2024-04-12T15:36:03Z |
| format | Article |
| id | doaj.art-c8e91b11b020445c85f64b3dc1cd4f2d |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-12T15:36:03Z |
| publishDate | 2006-07-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-c8e91b11b020445c85f64b3dc1cd4f2d2022-12-22T03:26:57ZengBMCBMC Medical Research Methodology1471-22882006-07-01613310.1186/1471-2288-6-33An alternative to the hand searching gold standard: validating methodological search filters using relative recallBarrowman Nicholas JMorrison AndraZhang LiSampson MargaretClifford Tammy JPlatt Robert WKlassen Terry PMoher David<p>Abstract</p> <p>Background</p> <p>Search filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard" – a reference standard against which to establish the performance of the filter. We demonstrate the feasibility of using relative recall of included studies from multiple systematic reviews to validate methodological search filters as an alternative to validation against a gold standard formed through hand searching.</p> <p>Methods</p> <p>We identified 105 Cochrane reviews that used the Highly Sensitive Search Strategy (HSSS), included randomized or quasi-randomized controlled trials, and reported their included studies. We measured the ability of two published and one novel variant of the HSSS to retrieve the MEDLINE-index studies included in these reviews.</p> <p>Results</p> <p>The systematic reviews were comprehensive in their searches. 72% of included primary studies were indexed in MEDLINE. Relative recall of the three strategies ranged from .98 to .91 across all reviews and more comprehensive strategies showed higher recall.</p> <p>Conclusion</p> <p>An approach using relative recall instead of a hand searching gold standard proved feasible and produced recall figures that were congruent with previously published figures for the HSSS. This technique would permit validation of a methodological filter using a collection of approximately 100 studies of the chosen design drawn from the included studies of multiple systematic reviews that used comprehensive search strategies.</p>http://www.biomedcentral.com/1471-2288/6/33 |
| spellingShingle | Barrowman Nicholas J Morrison Andra Zhang Li Sampson Margaret Clifford Tammy J Platt Robert W Klassen Terry P Moher David An alternative to the hand searching gold standard: validating methodological search filters using relative recall BMC Medical Research Methodology |
| title | An alternative to the hand searching gold standard: validating methodological search filters using relative recall |
| title_full | An alternative to the hand searching gold standard: validating methodological search filters using relative recall |
| title_fullStr | An alternative to the hand searching gold standard: validating methodological search filters using relative recall |
| title_full_unstemmed | An alternative to the hand searching gold standard: validating methodological search filters using relative recall |
| title_short | An alternative to the hand searching gold standard: validating methodological search filters using relative recall |
| title_sort | alternative to the hand searching gold standard validating methodological search filters using relative recall |
| url | http://www.biomedcentral.com/1471-2288/6/33 |
| work_keys_str_mv | AT barrowmannicholasj analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT morrisonandra analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT zhangli analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT sampsonmargaret analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT cliffordtammyj analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT plattrobertw analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT klassenterryp analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT moherdavid analternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT barrowmannicholasj alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT morrisonandra alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT zhangli alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT sampsonmargaret alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT cliffordtammyj alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT plattrobertw alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT klassenterryp alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall AT moherdavid alternativetothehandsearchinggoldstandardvalidatingmethodologicalsearchfiltersusingrelativerecall |