A systematic review of the content of critical appraisal tools
<p>Abstract</p> <p>Background</p> <p>Consumers of research (researchers, administrators, educators and clinicians) frequently use standard critical appraisal tools to evaluate the quality of published research reports. However, there is no consensus regarding the most a...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2004-09-01
|
| Series: | BMC Medical Research Methodology |
| Online Access: | http://www.biomedcentral.com/1471-2288/4/22 |
| _version_ | 1811311443972194304 |
|---|---|
| author | Kumar VS Saravana Massy-Westropp Nicola Bialocerkowski Andrea E Katrak Persis Grimmer Karen A |
| author_facet | Kumar VS Saravana Massy-Westropp Nicola Bialocerkowski Andrea E Katrak Persis Grimmer Karen A |
| author_sort | Kumar VS Saravana |
| collection | DOAJ |
| description | <p>Abstract</p> <p>Background</p> <p>Consumers of research (researchers, administrators, educators and clinicians) frequently use standard critical appraisal tools to evaluate the quality of published research reports. However, there is no consensus regarding the most appropriate critical appraisal tool for allied health research. We summarized the content, intent, construction and psychometric properties of published, currently available critical appraisal tools to identify common elements and their relevance to allied health research.</p> <p>Methods</p> <p>A systematic review was undertaken of 121 published critical appraisal tools sourced from 108 papers located on electronic databases and the Internet. The tools were classified according to the study design for which they were intended. Their items were then classified into one of 12 criteria based on their intent. Commonly occurring items were identified. The empirical basis for construction of the tool, the method by which overall quality of the study was established, the psychometric properties of the critical appraisal tools and whether guidelines were provided for their use were also recorded.</p> <p>Results</p> <p>Eighty-seven percent of critical appraisal tools were specific to a research design, with most tools having been developed for experimental studies. There was considerable variability in items contained in the critical appraisal tools. Twelve percent of available tools were developed using specified empirical research. Forty-nine percent of the critical appraisal tools summarized the quality appraisal into a numeric summary score. Few critical appraisal tools had documented evidence of validity of their items, or reliability of use. Guidelines regarding administration of the tools were provided in 43% of cases.</p> <p>Conclusions</p> <p>There was considerable variability in intent, components, construction and psychometric properties of published critical appraisal tools for research reports. There is no "gold standard' critical appraisal tool for any study design, nor is there any widely accepted generic tool that can be applied equally well across study types. No tool was specific to allied health research requirements. Thus interpretation of critical appraisal of research reports currently needs to be considered in light of the properties and intent of the critical appraisal tool chosen for the task.</p> |
| first_indexed | 2024-04-13T10:17:59Z |
| format | Article |
| id | doaj.art-e974a979ffd24a7091ae7b838532dfda |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-13T10:17:59Z |
| publishDate | 2004-09-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-e974a979ffd24a7091ae7b838532dfda2022-12-22T02:50:39ZengBMCBMC Medical Research Methodology1471-22882004-09-01412210.1186/1471-2288-4-22A systematic review of the content of critical appraisal toolsKumar VS SaravanaMassy-Westropp NicolaBialocerkowski Andrea EKatrak PersisGrimmer Karen A<p>Abstract</p> <p>Background</p> <p>Consumers of research (researchers, administrators, educators and clinicians) frequently use standard critical appraisal tools to evaluate the quality of published research reports. However, there is no consensus regarding the most appropriate critical appraisal tool for allied health research. We summarized the content, intent, construction and psychometric properties of published, currently available critical appraisal tools to identify common elements and their relevance to allied health research.</p> <p>Methods</p> <p>A systematic review was undertaken of 121 published critical appraisal tools sourced from 108 papers located on electronic databases and the Internet. The tools were classified according to the study design for which they were intended. Their items were then classified into one of 12 criteria based on their intent. Commonly occurring items were identified. The empirical basis for construction of the tool, the method by which overall quality of the study was established, the psychometric properties of the critical appraisal tools and whether guidelines were provided for their use were also recorded.</p> <p>Results</p> <p>Eighty-seven percent of critical appraisal tools were specific to a research design, with most tools having been developed for experimental studies. There was considerable variability in items contained in the critical appraisal tools. Twelve percent of available tools were developed using specified empirical research. Forty-nine percent of the critical appraisal tools summarized the quality appraisal into a numeric summary score. Few critical appraisal tools had documented evidence of validity of their items, or reliability of use. Guidelines regarding administration of the tools were provided in 43% of cases.</p> <p>Conclusions</p> <p>There was considerable variability in intent, components, construction and psychometric properties of published critical appraisal tools for research reports. There is no "gold standard' critical appraisal tool for any study design, nor is there any widely accepted generic tool that can be applied equally well across study types. No tool was specific to allied health research requirements. Thus interpretation of critical appraisal of research reports currently needs to be considered in light of the properties and intent of the critical appraisal tool chosen for the task.</p>http://www.biomedcentral.com/1471-2288/4/22 |
| spellingShingle | Kumar VS Saravana Massy-Westropp Nicola Bialocerkowski Andrea E Katrak Persis Grimmer Karen A A systematic review of the content of critical appraisal tools BMC Medical Research Methodology |
| title | A systematic review of the content of critical appraisal tools |
| title_full | A systematic review of the content of critical appraisal tools |
| title_fullStr | A systematic review of the content of critical appraisal tools |
| title_full_unstemmed | A systematic review of the content of critical appraisal tools |
| title_short | A systematic review of the content of critical appraisal tools |
| title_sort | systematic review of the content of critical appraisal tools |
| url | http://www.biomedcentral.com/1471-2288/4/22 |
| work_keys_str_mv | AT kumarvssaravana asystematicreviewofthecontentofcriticalappraisaltools AT massywestroppnicola asystematicreviewofthecontentofcriticalappraisaltools AT bialocerkowskiandreae asystematicreviewofthecontentofcriticalappraisaltools AT katrakpersis asystematicreviewofthecontentofcriticalappraisaltools AT grimmerkarena asystematicreviewofthecontentofcriticalappraisaltools AT kumarvssaravana systematicreviewofthecontentofcriticalappraisaltools AT massywestroppnicola systematicreviewofthecontentofcriticalappraisaltools AT bialocerkowskiandreae systematicreviewofthecontentofcriticalappraisaltools AT katrakpersis systematicreviewofthecontentofcriticalappraisaltools AT grimmerkarena systematicreviewofthecontentofcriticalappraisaltools |