The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys
It is important to evaluate risk of bias of the primary studies included in systematic reviews and meta-analyses. Since tools pertinent to surveys are scarce, the goal of the current research was to develop a measure to address this need. In Study 1, an initial list of 10 relevant topics was compile...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PsychOpen GOLD/ Leibniz Institute for Psychology
2020-12-01
|
| Series: | Methodology |
| Subjects: | |
| Online Access: | https://meth.psychopen.eu/index.php/meth/article/view/4329 |
| _version_ | 1828079595102928896 |
|---|---|
| author | Gabriel Nudelman Kathleen Otto |
| author_facet | Gabriel Nudelman Kathleen Otto |
| author_sort | Gabriel Nudelman |
| collection | DOAJ |
| description | It is important to evaluate risk of bias of the primary studies included in systematic reviews and meta-analyses. Since tools pertinent to surveys are scarce, the goal of the current research was to develop a measure to address this need. In Study 1, an initial list of 10 relevant topics was compiled from previous measures. In Study 2, the list was refined into an eight-item risk-of-bias measure via discussion and a pilot study. In Study 3, experienced researchers used the measure to asses 70 studies, demonstrating high interrater agreement (weighted Kappa = .82). Inexperienced raters also utilized the measure to code 26 different studies included in a prior meta-analysis, which resulted in adequate interrater agreement (weighted Kappa = .64) and excellent convergent validity (r = .66). Thus, the new measure, designed to be accessible and flexible, can increase standardization of risk-of-bias evaluations and contribute to the interpretation of systematic reviews and meta-analytic findings. |
| first_indexed | 2024-04-11T03:03:47Z |
| format | Article |
| id | doaj.art-4848c5bc61d74846a82477fd6d8489a7 |
| institution | Directory Open Access Journal |
| issn | 1614-2241 |
| language | English |
| last_indexed | 2024-04-11T03:03:47Z |
| publishDate | 2020-12-01 |
| publisher | PsychOpen GOLD/ Leibniz Institute for Psychology |
| record_format | Article |
| series | Methodology |
| spelling | doaj.art-4848c5bc61d74846a82477fd6d8489a72023-01-02T13:26:03ZengPsychOpen GOLD/ Leibniz Institute for PsychologyMethodology1614-22412020-12-0116427829810.5964/meth.4329meth.4329The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of SurveysGabriel Nudelman0Kathleen Otto1School of Behavioral Sciences, The Academic College of Tel Aviv-Yaffo, Jaffa, IsraelFaculty of Psychology, Philipps University of Marburg, Marburg, GermanyIt is important to evaluate risk of bias of the primary studies included in systematic reviews and meta-analyses. Since tools pertinent to surveys are scarce, the goal of the current research was to develop a measure to address this need. In Study 1, an initial list of 10 relevant topics was compiled from previous measures. In Study 2, the list was refined into an eight-item risk-of-bias measure via discussion and a pilot study. In Study 3, experienced researchers used the measure to asses 70 studies, demonstrating high interrater agreement (weighted Kappa = .82). Inexperienced raters also utilized the measure to code 26 different studies included in a prior meta-analysis, which resulted in adequate interrater agreement (weighted Kappa = .64) and excellent convergent validity (r = .66). Thus, the new measure, designed to be accessible and flexible, can increase standardization of risk-of-bias evaluations and contribute to the interpretation of systematic reviews and meta-analytic findings.https://meth.psychopen.eu/index.php/meth/article/view/4329risk of biassystematic reviewmeta-analysissurvey qualitycorrelational studies |
| spellingShingle | Gabriel Nudelman Kathleen Otto The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys Methodology risk of bias systematic review meta-analysis survey quality correlational studies |
| title | The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys |
| title_full | The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys |
| title_fullStr | The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys |
| title_full_unstemmed | The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys |
| title_short | The Development of a New Generic Risk-of-Bias Measure for Systematic Reviews of Surveys |
| title_sort | development of a new generic risk of bias measure for systematic reviews of surveys |
| topic | risk of bias systematic review meta-analysis survey quality correlational studies |
| url | https://meth.psychopen.eu/index.php/meth/article/view/4329 |
| work_keys_str_mv | AT gabrielnudelman thedevelopmentofanewgenericriskofbiasmeasureforsystematicreviewsofsurveys AT kathleenotto thedevelopmentofanewgenericriskofbiasmeasureforsystematicreviewsofsurveys AT gabrielnudelman developmentofanewgenericriskofbiasmeasureforsystematicreviewsofsurveys AT kathleenotto developmentofanewgenericriskofbiasmeasureforsystematicreviewsofsurveys |