Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH)
Objective There is no standard tool for assessing risk of bias (RoB) in prevalence studies. For the purposes of a living systematic review during the COVID-19 pandemic, we developed a tool to evaluate RoB in studies measuring the prevalence of mental health disorders (RoB-PrevMH) and tested inter-ra...
| Main Authors: | , , , , , , , , , |
|---|---|
| 格式: | 文件 |
| 语言: | English |
| 出版: |
BMJ Publishing Group
2023-10-01
|
| 丛编: | BMJ Mental Health |
| 在线阅读: | https://ebmh.bmj.com/content/26/1/e300694.full |
| _version_ | 1827392003662413824 |
|---|---|
| author | Nicola Low Toshi A Furukawa Andrea Cipriani Tianjing Li Georgia Salanti Stefan Leucht Diana Buitrago-Garcia Thomy Tonia Natalie Luise Peter Cristina Mesa-Vieira |
| author_facet | Nicola Low Toshi A Furukawa Andrea Cipriani Tianjing Li Georgia Salanti Stefan Leucht Diana Buitrago-Garcia Thomy Tonia Natalie Luise Peter Cristina Mesa-Vieira |
| author_sort | Nicola Low |
| collection | DOAJ |
| description | Objective There is no standard tool for assessing risk of bias (RoB) in prevalence studies. For the purposes of a living systematic review during the COVID-19 pandemic, we developed a tool to evaluate RoB in studies measuring the prevalence of mental health disorders (RoB-PrevMH) and tested inter-rater reliability.Methods We decided on items and signalling questions to include in RoB-PrevMH through iterative discussions. We tested the reliability of assessments by different users with two sets of prevalence studies. The first set included a random sample of 50 studies from our living systematic review. The second set included 33 studies from a systematic review of the prevalence of post-traumatic stress disorders, major depression and generalised anxiety disorder. We assessed the inter-rater agreement by calculating the proportion of agreement and Kappa statistic for each item.Results RoB-PrevMH consists of three items that address selection bias and information bias. Introductory and signalling questions guide the application of the tool to the review question. The inter-rater agreement for the three items was 83%, 90% and 93%. The weighted kappa scores were 0.63 (95% CI 0.54 to 0.73), 0.71 (95% CI 0.67 to 0.85) and 0.32 (95% CI −0.04 to 0.63), respectively.Conclusions RoB-PrevMH is a brief, user-friendly and adaptable tool for assessing RoB in studies on prevalence of mental health disorders. Initial results for inter-rater agreement were fair to substantial. The tool’s validity, reliability and applicability should be assessed in future projects. |
| first_indexed | 2024-03-08T17:23:38Z |
| format | Article |
| id | doaj.art-527b0b4a93b14cf880da6e5b89888cf5 |
| institution | Directory Open Access Journal |
| issn | 2755-9734 |
| language | English |
| last_indexed | 2024-03-08T17:23:38Z |
| publishDate | 2023-10-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Mental Health |
| spelling | doaj.art-527b0b4a93b14cf880da6e5b89888cf52024-01-03T02:45:09ZengBMJ Publishing GroupBMJ Mental Health2755-97342023-10-0126110.1136/bmjment-2023-300694Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH)Nicola Low0Toshi A Furukawa1Andrea Cipriani2Tianjing Li3Georgia Salanti4Stefan Leucht5Diana Buitrago-Garcia6Thomy Tonia7Natalie Luise Peter8Cristina Mesa-Vieira9Institute of Social and Preventive Medicine, University of Bern, Bern, SwitzerlandDepartment of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto University, Kyoto, Japan3 Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UKDepartment of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USAInstitute of Social and Preventive Medicine, University of Bern, Bern, SwitzerlandDepartment of Psychiatry and Psychotherapy, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Freising, GermanyInstitute of Social and Preventive Medicine, University of Bern, Bern, SwitzerlandInstitute of Social and Preventive Medicine, University of Bern, Bern, SwitzerlandDepartment of Psychiatry and Psychotherapy, Klinikum rechts der Isar, School of Medicine and Health, Technical University of Munich, München, GermanyInstitute of Social and Preventive Medicine, University of Bern, Bern, SwitzerlandObjective There is no standard tool for assessing risk of bias (RoB) in prevalence studies. For the purposes of a living systematic review during the COVID-19 pandemic, we developed a tool to evaluate RoB in studies measuring the prevalence of mental health disorders (RoB-PrevMH) and tested inter-rater reliability.Methods We decided on items and signalling questions to include in RoB-PrevMH through iterative discussions. We tested the reliability of assessments by different users with two sets of prevalence studies. The first set included a random sample of 50 studies from our living systematic review. The second set included 33 studies from a systematic review of the prevalence of post-traumatic stress disorders, major depression and generalised anxiety disorder. We assessed the inter-rater agreement by calculating the proportion of agreement and Kappa statistic for each item.Results RoB-PrevMH consists of three items that address selection bias and information bias. Introductory and signalling questions guide the application of the tool to the review question. The inter-rater agreement for the three items was 83%, 90% and 93%. The weighted kappa scores were 0.63 (95% CI 0.54 to 0.73), 0.71 (95% CI 0.67 to 0.85) and 0.32 (95% CI −0.04 to 0.63), respectively.Conclusions RoB-PrevMH is a brief, user-friendly and adaptable tool for assessing RoB in studies on prevalence of mental health disorders. Initial results for inter-rater agreement were fair to substantial. The tool’s validity, reliability and applicability should be assessed in future projects.https://ebmh.bmj.com/content/26/1/e300694.full |
| spellingShingle | Nicola Low Toshi A Furukawa Andrea Cipriani Tianjing Li Georgia Salanti Stefan Leucht Diana Buitrago-Garcia Thomy Tonia Natalie Luise Peter Cristina Mesa-Vieira Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH) BMJ Mental Health |
| title | Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH) |
| title_full | Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH) |
| title_fullStr | Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH) |
| title_full_unstemmed | Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH) |
| title_short | Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH) |
| title_sort | tool to assess risk of bias in studies estimating the prevalence of mental health disorders rob prevmh |
| url | https://ebmh.bmj.com/content/26/1/e300694.full |
| work_keys_str_mv | AT nicolalow tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT toshiafurukawa tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT andreacipriani tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT tianjingli tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT georgiasalanti tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT stefanleucht tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT dianabuitragogarcia tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT thomytonia tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT natalieluisepeter tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh AT cristinamesavieira tooltoassessriskofbiasinstudiesestimatingtheprevalenceofmentalhealthdisordersrobprevmh |