Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries
Abstract Background Existing implementation measures developed in high-income countries may have limited appropriateness for use within low- and middle-income countries (LMIC). In response, researchers at Johns Hopkins University began developing the Mental Health Implementation Science Tools (mhIST...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-05-01
|
| Series: | Implementation Science Communications |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s43058-022-00301-6 |
| _version_ | 1818548724632125440 |
|---|---|
| author | Luke R. Aldridge Christopher G. Kemp Judith K. Bass Kristen Danforth Jeremy C. Kane Syed U. Hamdani Lisa A. Marsch José M. Uribe-Restrepo Amanda J. Nguyen Paul A. Bolton Laura K. Murray Emily E. Haroz |
| author_facet | Luke R. Aldridge Christopher G. Kemp Judith K. Bass Kristen Danforth Jeremy C. Kane Syed U. Hamdani Lisa A. Marsch José M. Uribe-Restrepo Amanda J. Nguyen Paul A. Bolton Laura K. Murray Emily E. Haroz |
| author_sort | Luke R. Aldridge |
| collection | DOAJ |
| description | Abstract Background Existing implementation measures developed in high-income countries may have limited appropriateness for use within low- and middle-income countries (LMIC). In response, researchers at Johns Hopkins University began developing the Mental Health Implementation Science Tools (mhIST) in 2013 to assess priority implementation determinants and outcomes across four key stakeholder groups—consumers, providers, organization leaders, and policy makers—with dedicated versions of scales for each group. These were field tested and refined in several contexts, and criterion validity was established in Ukraine. The Consumer and Provider mhIST have since grown in popularity in mental health research, outpacing psychometric evaluation. Our objective was to establish the cross-context psychometric properties of these versions and inform future revisions. Methods We compiled secondary data from seven studies across six LMIC—Colombia, Myanmar, Pakistan, Thailand, Ukraine, and Zambia—to evaluate the psychometric performance of the Consumer and Provider mhIST. We used exploratory factor analysis to identify dimensionality, factor structure, and item loadings for each scale within each stakeholder version. We also used alignment analysis (i.e., multi-group confirmatory factor analysis) to estimate measurement invariance and differential item functioning of the Consumer scales across the six countries. Results All but one scale within the Provider and Consumer versions had Cronbach’s alpha greater than 0.8. Exploratory factor analysis indicated most scales were multidimensional, with factors generally aligning with a priori subscales for the Provider version; the Consumer version has no predefined subscales. Alignment analysis of the Consumer mhIST indicated a range of measurement invariance for scales across settings (R 2 0.46 to 0.77). Several items were identified for potential revision due to participant nonresponse or low or cross- factor loadings. We found only one item, which asked consumers whether their intervention provider was available when needed, to have differential item functioning in both intercept and loading. Conclusion We provide evidence that the Consumer and Provider versions of the mhIST are internally valid and reliable across diverse contexts and stakeholder groups for mental health research in LMIC. We recommend the instrument be revised based on these analyses and future research examine instrument utility by linking measurement to other outcomes of interest. |
| first_indexed | 2024-12-12T08:24:00Z |
| format | Article |
| id | doaj.art-730963b14c29463694f2ee692c01362d |
| institution | Directory Open Access Journal |
| issn | 2662-2211 |
| language | English |
| last_indexed | 2024-12-12T08:24:00Z |
| publishDate | 2022-05-01 |
| publisher | BMC |
| record_format | Article |
| series | Implementation Science Communications |
| spelling | doaj.art-730963b14c29463694f2ee692c01362d2022-12-22T00:31:18ZengBMCImplementation Science Communications2662-22112022-05-013111410.1186/s43058-022-00301-6Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countriesLuke R. Aldridge0Christopher G. Kemp1Judith K. Bass2Kristen Danforth3Jeremy C. Kane4Syed U. Hamdani5Lisa A. Marsch6José M. Uribe-Restrepo7Amanda J. Nguyen8Paul A. Bolton9Laura K. Murray10Emily E. Haroz11Johns Hopkins University Bloomberg School of Public HealthJohns Hopkins University Bloomberg School of Public HealthJohns Hopkins University Bloomberg School of Public HealthUniversity of Washington Department of Global HealthColumbia University Mailman School of Public HealthUniversity of Liverpool Institute of Population HealthDartmouth Center for Technology & Behavioral HealthPontificia Universidad Javeriana Department of Psychiatry and Mental HealthUniversity of Virginia School of Education and Human DevelopmentJohns Hopkins University Bloomberg School of Public HealthJohns Hopkins University Bloomberg School of Public HealthJohns Hopkins University Bloomberg School of Public HealthAbstract Background Existing implementation measures developed in high-income countries may have limited appropriateness for use within low- and middle-income countries (LMIC). In response, researchers at Johns Hopkins University began developing the Mental Health Implementation Science Tools (mhIST) in 2013 to assess priority implementation determinants and outcomes across four key stakeholder groups—consumers, providers, organization leaders, and policy makers—with dedicated versions of scales for each group. These were field tested and refined in several contexts, and criterion validity was established in Ukraine. The Consumer and Provider mhIST have since grown in popularity in mental health research, outpacing psychometric evaluation. Our objective was to establish the cross-context psychometric properties of these versions and inform future revisions. Methods We compiled secondary data from seven studies across six LMIC—Colombia, Myanmar, Pakistan, Thailand, Ukraine, and Zambia—to evaluate the psychometric performance of the Consumer and Provider mhIST. We used exploratory factor analysis to identify dimensionality, factor structure, and item loadings for each scale within each stakeholder version. We also used alignment analysis (i.e., multi-group confirmatory factor analysis) to estimate measurement invariance and differential item functioning of the Consumer scales across the six countries. Results All but one scale within the Provider and Consumer versions had Cronbach’s alpha greater than 0.8. Exploratory factor analysis indicated most scales were multidimensional, with factors generally aligning with a priori subscales for the Provider version; the Consumer version has no predefined subscales. Alignment analysis of the Consumer mhIST indicated a range of measurement invariance for scales across settings (R 2 0.46 to 0.77). Several items were identified for potential revision due to participant nonresponse or low or cross- factor loadings. We found only one item, which asked consumers whether their intervention provider was available when needed, to have differential item functioning in both intercept and loading. Conclusion We provide evidence that the Consumer and Provider versions of the mhIST are internally valid and reliable across diverse contexts and stakeholder groups for mental health research in LMIC. We recommend the instrument be revised based on these analyses and future research examine instrument utility by linking measurement to other outcomes of interest.https://doi.org/10.1186/s43058-022-00301-6Mental healthImplementation measurementPsychometricsLow- and middle-income countries |
| spellingShingle | Luke R. Aldridge Christopher G. Kemp Judith K. Bass Kristen Danforth Jeremy C. Kane Syed U. Hamdani Lisa A. Marsch José M. Uribe-Restrepo Amanda J. Nguyen Paul A. Bolton Laura K. Murray Emily E. Haroz Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries Implementation Science Communications Mental health Implementation measurement Psychometrics Low- and middle-income countries |
| title | Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries |
| title_full | Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries |
| title_fullStr | Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries |
| title_full_unstemmed | Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries |
| title_short | Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries |
| title_sort | psychometric performance of the mental health implementation science tools mhist across six low and middle income countries |
| topic | Mental health Implementation measurement Psychometrics Low- and middle-income countries |
| url | https://doi.org/10.1186/s43058-022-00301-6 |
| work_keys_str_mv | AT lukeraldridge psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT christophergkemp psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT judithkbass psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT kristendanforth psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT jeremyckane psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT syeduhamdani psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT lisaamarsch psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT josemuriberestrepo psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT amandajnguyen psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT paulabolton psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT laurakmurray psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries AT emilyeharoz psychometricperformanceofthementalhealthimplementationsciencetoolsmhistacrosssixlowandmiddleincomecountries |