Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC)
Abstract In this letter, we briefly describe how we selected and implemented the quality criteria checklist (QCC) as a critical appraisal tool in rapid systematic reviews conducted to inform public health advice, guidance and policy during the COVID-19 pandemic. As these rapid reviews usually includ...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-03-01
|
| Series: | Systematic Reviews |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13643-023-02219-z |
| _version_ | 1827974748516122624 |
|---|---|
| author | Daphne Duval Nicola Pearce-Smith Jennifer C. Palmer Jason Kwasi Sarfo-Annin Paul Rudd Rachel Clark |
| author_facet | Daphne Duval Nicola Pearce-Smith Jennifer C. Palmer Jason Kwasi Sarfo-Annin Paul Rudd Rachel Clark |
| author_sort | Daphne Duval |
| collection | DOAJ |
| description | Abstract In this letter, we briefly describe how we selected and implemented the quality criteria checklist (QCC) as a critical appraisal tool in rapid systematic reviews conducted to inform public health advice, guidance and policy during the COVID-19 pandemic. As these rapid reviews usually included a range of study designs, it was key to identify a single tool that would allow for reliable critical appraisal across most experimental and observational study designs and applicable to a range of topics. After carefully considering a number of existing tools, the QCC was selected as it had good interrater agreement between three reviewers (Fleiss kappa coefficient 0.639) and was found to be easy and fast to apply once familiar with the tool. The QCC consists of 10 questions, with sub-questions to specify how it should be applied to a specific study design. Four of these questions are considered as critical (on selection bias, group comparability, intervention/exposure assessment and outcome assessment) and the rating of a study (high, moderate or low methodological quality) depends on the responses to these four critical questions. Our results suggest that the QCC is an appropriate critical appraisal tool to assess experimental and observational studies within COVID-19 rapid reviews. This study was done at pace during the COVID-19 pandemic; further reliability analyses should be conducted, and more research is needed to validate the QCC across a range of public health topics. |
| first_indexed | 2024-04-09T19:59:20Z |
| format | Article |
| id | doaj.art-1e516018967343068813674cb4bb83f6 |
| institution | Directory Open Access Journal |
| issn | 2046-4053 |
| language | English |
| last_indexed | 2024-04-09T19:59:20Z |
| publishDate | 2023-03-01 |
| publisher | BMC |
| record_format | Article |
| series | Systematic Reviews |
| spelling | doaj.art-1e516018967343068813674cb4bb83f62023-04-03T05:18:15ZengBMCSystematic Reviews2046-40532023-03-011211410.1186/s13643-023-02219-zCritical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC)Daphne Duval0Nicola Pearce-Smith1Jennifer C. Palmer2Jason Kwasi Sarfo-Annin3Paul Rudd4Rachel Clark5COVID-19 Rapid Evidence Service, UK Health Security Agency (UKHSA)COVID-19 Rapid Evidence Service, UK Health Security Agency (UKHSA)COVID-19 Rapid Evidence Service, UK Health Security Agency (UKHSA)COVID-19 Rapid Evidence Service, UK Health Security Agency (UKHSA)COVID-19 Rapid Evidence Service, UK Health Security Agency (UKHSA)COVID-19 Rapid Evidence Service, UK Health Security Agency (UKHSA)Abstract In this letter, we briefly describe how we selected and implemented the quality criteria checklist (QCC) as a critical appraisal tool in rapid systematic reviews conducted to inform public health advice, guidance and policy during the COVID-19 pandemic. As these rapid reviews usually included a range of study designs, it was key to identify a single tool that would allow for reliable critical appraisal across most experimental and observational study designs and applicable to a range of topics. After carefully considering a number of existing tools, the QCC was selected as it had good interrater agreement between three reviewers (Fleiss kappa coefficient 0.639) and was found to be easy and fast to apply once familiar with the tool. The QCC consists of 10 questions, with sub-questions to specify how it should be applied to a specific study design. Four of these questions are considered as critical (on selection bias, group comparability, intervention/exposure assessment and outcome assessment) and the rating of a study (high, moderate or low methodological quality) depends on the responses to these four critical questions. Our results suggest that the QCC is an appropriate critical appraisal tool to assess experimental and observational studies within COVID-19 rapid reviews. This study was done at pace during the COVID-19 pandemic; further reliability analyses should be conducted, and more research is needed to validate the QCC across a range of public health topics.https://doi.org/10.1186/s13643-023-02219-zCritical appraisalRisk of bias toolRapid systematic reviewsPublic health |
| spellingShingle | Daphne Duval Nicola Pearce-Smith Jennifer C. Palmer Jason Kwasi Sarfo-Annin Paul Rudd Rachel Clark Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC) Systematic Reviews Critical appraisal Risk of bias tool Rapid systematic reviews Public health |
| title | Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC) |
| title_full | Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC) |
| title_fullStr | Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC) |
| title_full_unstemmed | Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC) |
| title_short | Critical appraisal in rapid systematic reviews of COVID-19 studies: implementation of the Quality Criteria Checklist (QCC) |
| title_sort | critical appraisal in rapid systematic reviews of covid 19 studies implementation of the quality criteria checklist qcc |
| topic | Critical appraisal Risk of bias tool Rapid systematic reviews Public health |
| url | https://doi.org/10.1186/s13643-023-02219-z |
| work_keys_str_mv | AT daphneduval criticalappraisalinrapidsystematicreviewsofcovid19studiesimplementationofthequalitycriteriachecklistqcc AT nicolapearcesmith criticalappraisalinrapidsystematicreviewsofcovid19studiesimplementationofthequalitycriteriachecklistqcc AT jennifercpalmer criticalappraisalinrapidsystematicreviewsofcovid19studiesimplementationofthequalitycriteriachecklistqcc AT jasonkwasisarfoannin criticalappraisalinrapidsystematicreviewsofcovid19studiesimplementationofthequalitycriteriachecklistqcc AT paulrudd criticalappraisalinrapidsystematicreviewsofcovid19studiesimplementationofthequalitycriteriachecklistqcc AT rachelclark criticalappraisalinrapidsystematicreviewsofcovid19studiesimplementationofthequalitycriteriachecklistqcc |