Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis
We aimed to find agreement between diagnoses obtained through standardized (SDI) and non-standardized diagnostic interviews (NSDI) for schizophrenia and Bipolar Affective Disorder (BD). Methods: A systematic review with meta-analysis was conducted. Publications from 2007 to 2020 comparing SDI and NS...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Diagnostics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4418/13/3/526 |
| _version_ | 1797624818576654336 |
|---|---|
| author | Hélio Rocha Neto Ana Lúcia R. Moreira Lucas Hosken Joshua A. Langfus Maria Tavares Cavalcanti Eric Arden Youngstrom Diogo Telles-Correia |
| author_facet | Hélio Rocha Neto Ana Lúcia R. Moreira Lucas Hosken Joshua A. Langfus Maria Tavares Cavalcanti Eric Arden Youngstrom Diogo Telles-Correia |
| author_sort | Hélio Rocha Neto |
| collection | DOAJ |
| description | We aimed to find agreement between diagnoses obtained through standardized (SDI) and non-standardized diagnostic interviews (NSDI) for schizophrenia and Bipolar Affective Disorder (BD). Methods: A systematic review with meta-analysis was conducted. Publications from 2007 to 2020 comparing SDI and NSDI diagnoses in adults without neurological disorders were screened in MEDLINE, ISI Web of Science, and SCOPUS, following PROSPERO registration CRD42020187157, PRISMA guidelines, and quality assessment using QUADAS–2. Results: From 54231 entries, 22 studies were analyzed, and 13 were included in the final meta-analysis of kappa agreement using a mixed-effects meta-regression model. A mean kappa of 0.41 (Fair agreement, 95% CI: 0.34 to 0.47) but high heterogeneity (<i>Î</i><sup>2</sup> = 92%) were calculated. Gender, mean age, NSDI setting (Inpatient vs. Outpatient; University vs. Non-university), and SDI informant (Self vs. Professional) were tested as predictors in meta-regression. Only SDI informant was relevant for the explanatory model, leaving 79% unexplained heterogeneity. Egger’s test did not indicate significant bias, and QUADAS–2 resulted in “average” data quality. Conclusions: Most studies using SDIs do not report the original sample size, only the SDI-diagnosed patients. Kappa comparison resulted in high heterogeneity, which may reflect the influence of non-systematic bias in diagnostic processes. Although results were highly heterogeneous, we measured a fair agreement kappa between SDI and NSDI, implying clinicians might operate in scenarios not equivalent to psychiatry trials, where samples are filtered, and there may be more emphasis on maintaining reliability. The present study received no funding. |
| first_indexed | 2024-03-11T09:48:04Z |
| format | Article |
| id | doaj.art-5be6f831650249c3a96ee94722ce86a8 |
| institution | Directory Open Access Journal |
| issn | 2075-4418 |
| language | English |
| last_indexed | 2024-03-11T09:48:04Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Diagnostics |
| spelling | doaj.art-5be6f831650249c3a96ee94722ce86a82023-11-16T16:26:04ZengMDPI AGDiagnostics2075-44182023-01-0113352610.3390/diagnostics13030526Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-AnalysisHélio Rocha Neto0Ana Lúcia R. Moreira1Lucas Hosken2Joshua A. Langfus3Maria Tavares Cavalcanti4Eric Arden Youngstrom5Diogo Telles-Correia6Medical Faculty, Lisbon University, 1649-028 Lisbon, PortugalCork Kerry Community Healthcare, T12YE02 Cork, IrelandMedical Psychology Sector, University Hospital Clementino Fraga Filho, HUCFF/UFRJ, Rio de Janeiro 21941913, RJ, BrazilClinical Psychology Program, Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-3270, USAPrograma de Pós Graduação em Psiquiatria e Saúde Mental—PROPSAM, Instituto de Psiquiatria, Universidade Federal do Rio de Janeiro—UFRJ, Rio de Janeiro 22290140, RJ, BrazilClinical Psychology Program, Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-3270, USAMedical Faculty, Lisbon University, 1649-028 Lisbon, PortugalWe aimed to find agreement between diagnoses obtained through standardized (SDI) and non-standardized diagnostic interviews (NSDI) for schizophrenia and Bipolar Affective Disorder (BD). Methods: A systematic review with meta-analysis was conducted. Publications from 2007 to 2020 comparing SDI and NSDI diagnoses in adults without neurological disorders were screened in MEDLINE, ISI Web of Science, and SCOPUS, following PROSPERO registration CRD42020187157, PRISMA guidelines, and quality assessment using QUADAS–2. Results: From 54231 entries, 22 studies were analyzed, and 13 were included in the final meta-analysis of kappa agreement using a mixed-effects meta-regression model. A mean kappa of 0.41 (Fair agreement, 95% CI: 0.34 to 0.47) but high heterogeneity (<i>Î</i><sup>2</sup> = 92%) were calculated. Gender, mean age, NSDI setting (Inpatient vs. Outpatient; University vs. Non-university), and SDI informant (Self vs. Professional) were tested as predictors in meta-regression. Only SDI informant was relevant for the explanatory model, leaving 79% unexplained heterogeneity. Egger’s test did not indicate significant bias, and QUADAS–2 resulted in “average” data quality. Conclusions: Most studies using SDIs do not report the original sample size, only the SDI-diagnosed patients. Kappa comparison resulted in high heterogeneity, which may reflect the influence of non-systematic bias in diagnostic processes. Although results were highly heterogeneous, we measured a fair agreement kappa between SDI and NSDI, implying clinicians might operate in scenarios not equivalent to psychiatry trials, where samples are filtered, and there may be more emphasis on maintaining reliability. The present study received no funding.https://www.mdpi.com/2075-4418/13/3/526standard diagnostic interviewmeta-analysisschizophreniabipolar affective disorderreliability and validity |
| spellingShingle | Hélio Rocha Neto Ana Lúcia R. Moreira Lucas Hosken Joshua A. Langfus Maria Tavares Cavalcanti Eric Arden Youngstrom Diogo Telles-Correia Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis Diagnostics standard diagnostic interview meta-analysis schizophrenia bipolar affective disorder reliability and validity |
| title | Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis |
| title_full | Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis |
| title_fullStr | Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis |
| title_full_unstemmed | Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis |
| title_short | Inter-Rater Reliability between Structured and Non-Structured Interviews Is Fair in Schizophrenia and Bipolar Disorders—A Systematic Review and Meta-Analysis |
| title_sort | inter rater reliability between structured and non structured interviews is fair in schizophrenia and bipolar disorders a systematic review and meta analysis |
| topic | standard diagnostic interview meta-analysis schizophrenia bipolar affective disorder reliability and validity |
| url | https://www.mdpi.com/2075-4418/13/3/526 |
| work_keys_str_mv | AT heliorochaneto interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis AT analuciarmoreira interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis AT lucashosken interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis AT joshuaalangfus interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis AT mariatavarescavalcanti interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis AT ericardenyoungstrom interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis AT diogotellescorreia interraterreliabilitybetweenstructuredandnonstructuredinterviewsisfairinschizophreniaandbipolardisordersasystematicreviewandmetaanalysis |