Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy
Abstract Background Synthesis of psychometric properties of substance use measures to identify patterns of use and substance use disorders remains limited. To address this gap, we sought to systematically evaluate the psychometric properties of measures to detect substance use and misuse. Methods We...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2020-05-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12874-020-00963-7 |
| _version_ | 1818061565380788224 |
|---|---|
| author | Glenn-Milo Santos Steffanie A. Strathdee Nabila El-Bassel Poonam Patel Divya Subramanian Danielle Horyniak Ryan R. Cook Charlotte McCullagh Phillip Marotta Foram Choksi Brian Kang Isabel Allen Steven Shoptaw |
| author_facet | Glenn-Milo Santos Steffanie A. Strathdee Nabila El-Bassel Poonam Patel Divya Subramanian Danielle Horyniak Ryan R. Cook Charlotte McCullagh Phillip Marotta Foram Choksi Brian Kang Isabel Allen Steven Shoptaw |
| author_sort | Glenn-Milo Santos |
| collection | DOAJ |
| description | Abstract Background Synthesis of psychometric properties of substance use measures to identify patterns of use and substance use disorders remains limited. To address this gap, we sought to systematically evaluate the psychometric properties of measures to detect substance use and misuse. Methods We conducted a systematic review and meta-analysis of literature on measures of substance classes associated with HIV risk (heroin, methamphetamine, cocaine, ecstasy, alcohol) that were published in English before June 2016 that reported at least one of the following psychometric outcomes of interest: internal consistency (alpha), test-retest/inter-rater reliability (kappa), sensitivity, specificity, positive predictive value, and negative predictive value. We used meta-analytic techniques to generate pooled summary estimates for these outcomes using random effects and hierarchical logistic regression models. Results Findings across 387 paper revealed that overall, 65% of pooled estimates for alpha were in the range of fair-to-excellent; 44% of estimates for kappa were in the range of fair-to-excellent. In addition, 69, 97, 37 and 96% of pooled estimates for sensitivity, specificity, positive predictive value, and negative predictive value, respectively, were in the range of moderate-to-excellent. Conclusion We conclude that many substance use measures had pooled summary estimates that were at the fair/moderate-to-excellent range across different psychometric outcomes. Most scales were conducted in English, within the United States, highlighting the need to test and validate these measures in more diverse settings. Additionally, the majority of studies had high risk of bias, indicating a need for more studies with higher methodological quality. |
| first_indexed | 2024-12-10T13:50:20Z |
| format | Article |
| id | doaj.art-41582a9c6ea94c2481acb2a95fccdbef |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-10T13:50:20Z |
| publishDate | 2020-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-41582a9c6ea94c2481acb2a95fccdbef2022-12-22T01:46:13ZengBMCBMC Medical Research Methodology1471-22882020-05-0120112210.1186/s12874-020-00963-7Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracyGlenn-Milo Santos0Steffanie A. Strathdee1Nabila El-Bassel2Poonam Patel3Divya Subramanian4Danielle Horyniak5Ryan R. Cook6Charlotte McCullagh7Phillip Marotta8Foram Choksi9Brian Kang10Isabel Allen11Steven Shoptaw12Department of Community Health Systems, University of California San FranciscoDivision of Global Public Health, University of California San DiegoSchool of Social Work, Columbia UniversityCenter for Public Health Research, San Francisco Department of Public HealthCenter for Public Health Research, San Francisco Department of Public HealthDivision of Global Public Health, University of California San DiegoDepartment of Family Medicine and Psychiatry and Biobehavioral Sciences, University of California Los AngelesSchool of Social Work, Columbia UniversitySchool of Social Work, Columbia UniversityCenter for Public Health Research, San Francisco Department of Public HealthCenter for Public Health Research, San Francisco Department of Public HealthDepartment of Epidemiology and Biostatistics, University of California San FranciscoDepartment of Family Medicine and Psychiatry and Biobehavioral Sciences, University of California Los AngelesAbstract Background Synthesis of psychometric properties of substance use measures to identify patterns of use and substance use disorders remains limited. To address this gap, we sought to systematically evaluate the psychometric properties of measures to detect substance use and misuse. Methods We conducted a systematic review and meta-analysis of literature on measures of substance classes associated with HIV risk (heroin, methamphetamine, cocaine, ecstasy, alcohol) that were published in English before June 2016 that reported at least one of the following psychometric outcomes of interest: internal consistency (alpha), test-retest/inter-rater reliability (kappa), sensitivity, specificity, positive predictive value, and negative predictive value. We used meta-analytic techniques to generate pooled summary estimates for these outcomes using random effects and hierarchical logistic regression models. Results Findings across 387 paper revealed that overall, 65% of pooled estimates for alpha were in the range of fair-to-excellent; 44% of estimates for kappa were in the range of fair-to-excellent. In addition, 69, 97, 37 and 96% of pooled estimates for sensitivity, specificity, positive predictive value, and negative predictive value, respectively, were in the range of moderate-to-excellent. Conclusion We conclude that many substance use measures had pooled summary estimates that were at the fair/moderate-to-excellent range across different psychometric outcomes. Most scales were conducted in English, within the United States, highlighting the need to test and validate these measures in more diverse settings. Additionally, the majority of studies had high risk of bias, indicating a need for more studies with higher methodological quality.http://link.springer.com/article/10.1186/s12874-020-00963-7Substance useAlcoholDrugsPsychometric propertiesMeta-analysis |
| spellingShingle | Glenn-Milo Santos Steffanie A. Strathdee Nabila El-Bassel Poonam Patel Divya Subramanian Danielle Horyniak Ryan R. Cook Charlotte McCullagh Phillip Marotta Foram Choksi Brian Kang Isabel Allen Steven Shoptaw Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy BMC Medical Research Methodology Substance use Alcohol Drugs Psychometric properties Meta-analysis |
| title | Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy |
| title_full | Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy |
| title_fullStr | Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy |
| title_full_unstemmed | Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy |
| title_short | Psychometric properties of measures of substance use: a systematic review and meta-analysis of reliability, validity and diagnostic test accuracy |
| title_sort | psychometric properties of measures of substance use a systematic review and meta analysis of reliability validity and diagnostic test accuracy |
| topic | Substance use Alcohol Drugs Psychometric properties Meta-analysis |
| url | http://link.springer.com/article/10.1186/s12874-020-00963-7 |
| work_keys_str_mv | AT glennmilosantos psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT steffanieastrathdee psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT nabilaelbassel psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT poonampatel psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT divyasubramanian psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT daniellehoryniak psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT ryanrcook psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT charlottemccullagh psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT phillipmarotta psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT foramchoksi psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT briankang psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT isabelallen psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy AT stevenshoptaw psychometricpropertiesofmeasuresofsubstanceuseasystematicreviewandmetaanalysisofreliabilityvalidityanddiagnostictestaccuracy |