User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary
Abstract Background ‘A Measurement Tool to Assess Systematic Reviews, version 2’ (AMSTAR 2) is a validated 16-item scale designed to appraise systematic reviews (SRs) of healthcare interventions and to rate the overall confidence in their results. This commentary aims to describe the challenges with...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-03-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-023-01879-8 |
| _version_ | 1797864330228662272 |
|---|---|
| author | Karina Karolina De Santis Dawid Pieper Robert C. Lorenz Uta Wegewitz Waldemar Siemens Katja Matthias |
| author_facet | Karina Karolina De Santis Dawid Pieper Robert C. Lorenz Uta Wegewitz Waldemar Siemens Katja Matthias |
| author_sort | Karina Karolina De Santis |
| collection | DOAJ |
| description | Abstract Background ‘A Measurement Tool to Assess Systematic Reviews, version 2’ (AMSTAR 2) is a validated 16-item scale designed to appraise systematic reviews (SRs) of healthcare interventions and to rate the overall confidence in their results. This commentary aims to describe the challenges with rating of the individual items and the application of AMSTAR 2 from the user perspective. Discussion A group of six experienced users (methodologists working in different clinical fields for at least 10 years) identified and discussed the challenges in rating of each item and the general use of AMSTAR 2 to appraise SRs. A group discussion was used to develop recommendations on how users could deal with the identified challenges. We identified various challenges with the content of items 2–16 and with the derivation of the overall confidence ratings on AMSTAR 2. These challenges include the need (1) to provide additional definitions (e.g., what constitutes major deviations from SR protocol on item 2), (2) to choose a rating strategy for multiple conditions on single items (e.g., how to rate item 5 if studies were selected in duplicate, but consensus between two authors was not reported), and (3) to determine rules for deriving the confidence ratings (e.g., what items are critical for such ratings). Based on these challenges we formulated specific recommendations for items 2–16 that AMSTAR 2 users could consider before applying the tool. Summary Our commentary adds to the existing literature by providing the first in-depth examination of the AMSTAR 2 tool from the user perspective. The identified challenges could be addressed by additional decision rules including definitions for ambiguous items and guidance for rating of complex items and derivation of confidence ratings. We recommend that a team consensus regarding such decision rules is required before appraisal procedure begins. Trial registration Not applicable. |
| first_indexed | 2024-04-09T22:50:04Z |
| format | Article |
| id | doaj.art-10849d7501d548c68d066f0e81f6aab4 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-09T22:50:04Z |
| publishDate | 2023-03-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-10849d7501d548c68d066f0e81f6aab42023-03-22T11:38:56ZengBMCBMC Medical Research Methodology1471-22882023-03-0123111010.1186/s12874-023-01879-8User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentaryKarina Karolina De Santis0Dawid Pieper1Robert C. Lorenz2Uta Wegewitz3Waldemar Siemens4Katja Matthias5Department of Prevention and Evaluation, Leibniz Institute for Prevention Research and Epidemiology – BIPS GmbHBrandenburg Medical School Theodor Fontane (MHB), Center for Health Services Research (ZVF-BB)Lise Meitner Group for Environmental Neuroscience, Max Planck Institute for Human DevelopmentFederal Institute for Occupational Safety and Health (BAuA), Division 3 Work and HealthFaculty of Medicine, Institute for Evidence in Medicine, Medical Center - University of Freiburg, University of FreiburgFaculty of Electrical Engineering and Computer Science, University of Applied Sciences StralsundAbstract Background ‘A Measurement Tool to Assess Systematic Reviews, version 2’ (AMSTAR 2) is a validated 16-item scale designed to appraise systematic reviews (SRs) of healthcare interventions and to rate the overall confidence in their results. This commentary aims to describe the challenges with rating of the individual items and the application of AMSTAR 2 from the user perspective. Discussion A group of six experienced users (methodologists working in different clinical fields for at least 10 years) identified and discussed the challenges in rating of each item and the general use of AMSTAR 2 to appraise SRs. A group discussion was used to develop recommendations on how users could deal with the identified challenges. We identified various challenges with the content of items 2–16 and with the derivation of the overall confidence ratings on AMSTAR 2. These challenges include the need (1) to provide additional definitions (e.g., what constitutes major deviations from SR protocol on item 2), (2) to choose a rating strategy for multiple conditions on single items (e.g., how to rate item 5 if studies were selected in duplicate, but consensus between two authors was not reported), and (3) to determine rules for deriving the confidence ratings (e.g., what items are critical for such ratings). Based on these challenges we formulated specific recommendations for items 2–16 that AMSTAR 2 users could consider before applying the tool. Summary Our commentary adds to the existing literature by providing the first in-depth examination of the AMSTAR 2 tool from the user perspective. The identified challenges could be addressed by additional decision rules including definitions for ambiguous items and guidance for rating of complex items and derivation of confidence ratings. We recommend that a team consensus regarding such decision rules is required before appraisal procedure begins. Trial registration Not applicable.https://doi.org/10.1186/s12874-023-01879-8AMSTAR 2Systematic review (SR)Evidence appraisalConfidence rating |
| spellingShingle | Karina Karolina De Santis Dawid Pieper Robert C. Lorenz Uta Wegewitz Waldemar Siemens Katja Matthias User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary BMC Medical Research Methodology AMSTAR 2 Systematic review (SR) Evidence appraisal Confidence rating |
| title | User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary |
| title_full | User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary |
| title_fullStr | User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary |
| title_full_unstemmed | User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary |
| title_short | User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary |
| title_sort | user experience of applying amstar 2 to appraise systematic reviews of healthcare interventions a commentary |
| topic | AMSTAR 2 Systematic review (SR) Evidence appraisal Confidence rating |
| url | https://doi.org/10.1186/s12874-023-01879-8 |
| work_keys_str_mv | AT karinakarolinadesantis userexperienceofapplyingamstar2toappraisesystematicreviewsofhealthcareinterventionsacommentary AT dawidpieper userexperienceofapplyingamstar2toappraisesystematicreviewsofhealthcareinterventionsacommentary AT robertclorenz userexperienceofapplyingamstar2toappraisesystematicreviewsofhealthcareinterventionsacommentary AT utawegewitz userexperienceofapplyingamstar2toappraisesystematicreviewsofhealthcareinterventionsacommentary AT waldemarsiemens userexperienceofapplyingamstar2toappraisesystematicreviewsofhealthcareinterventionsacommentary AT katjamatthias userexperienceofapplyingamstar2toappraisesystematicreviewsofhealthcareinterventionsacommentary |