Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study
BackgroundDigital mental health interventions are being used more than ever for the prevention and treatment of psychological problems. Optimizing the implementation aspects of digital mental health is essential to deliver the program to populations in need, but there is a la...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2021-11-01
|
| Series: | JMIR Formative Research |
| Online Access: | https://formative.jmir.org/2021/11/e24332 |
| _version_ | 1797735565476495360 |
|---|---|
| author | Natsu Sasaki Erika Obikane Rajesh Vedanthan Kotaro Imamura Pim Cuijpers Taichi Shimazu Masamitsu Kamada Norito Kawakami Daisuke Nishi |
| author_facet | Natsu Sasaki Erika Obikane Rajesh Vedanthan Kotaro Imamura Pim Cuijpers Taichi Shimazu Masamitsu Kamada Norito Kawakami Daisuke Nishi |
| author_sort | Natsu Sasaki |
| collection | DOAJ |
| description |
BackgroundDigital mental health interventions are being used more than ever for the prevention and treatment of psychological problems. Optimizing the implementation aspects of digital mental health is essential to deliver the program to populations in need, but there is a lack of validated implementation outcome measures for digital mental health interventions.
ObjectiveThe primary aim of this study is to develop implementation outcome scales of digital mental health for different levels of stakeholders involved in the implementation process: users, providers, and managers or policy makers. The secondary aim is to validate the developed scale for users.
MethodsWe developed English and Japanese versions of the implementation outcome scales for digital mental health (iOSDMH) based on the literature review and panel discussions with experts in implementation research and web-based psychotherapy. The study developed acceptability, appropriateness, feasibility, satisfaction, and harm as the outcome measures for users, providers, and managers or policy makers. We conducted evidence-based interventions via the internet using UTSMeD, a website for mental health information (N=200). Exploratory factor analysis (EFA) was conducted to assess the structural validity of the iOSDMH for users. Satisfaction, which consisted of a single item, was not included in the EFA.
ResultsThe iOSDMH was developed for users, providers, and managers or policy makers. The iOSDMH contains 19 items for users, 11 items for providers, and 14 items for managers or policy makers. Cronbach α coefficients indicated intermediate internal consistency for acceptability (α=.665) but high consistency for appropriateness (α=.776), feasibility (α=.832), and harm (α=.777) of the iOSDMH for users. EFA revealed 3-factor structures, indicating acceptability and appropriateness as close concepts. Despite the similarity between these 2 concepts, we inferred that acceptability and appropriateness should be used as different factors, following previous studies.
ConclusionsWe developed iOSDMH for users, providers, and managers. Psychometric assessment of the scales for users demonstrated acceptable reliability and validity. Evaluating the components of digital mental health implementation is a major step forward in implementation science. |
| first_indexed | 2024-03-12T13:00:54Z |
| format | Article |
| id | doaj.art-d552ea41dd764fc1baa52e0c8950af99 |
| institution | Directory Open Access Journal |
| issn | 2561-326X |
| language | English |
| last_indexed | 2024-03-12T13:00:54Z |
| publishDate | 2021-11-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Formative Research |
| spelling | doaj.art-d552ea41dd764fc1baa52e0c8950af992023-08-28T19:49:26ZengJMIR PublicationsJMIR Formative Research2561-326X2021-11-01511e2433210.2196/24332Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional StudyNatsu Sasakihttps://orcid.org/0000-0002-6097-5948Erika Obikanehttps://orcid.org/0000-0002-8251-8094Rajesh Vedanthanhttps://orcid.org/0000-0001-7138-2382Kotaro Imamurahttps://orcid.org/0000-0003-2584-0666Pim Cuijpershttps://orcid.org/0000-0001-5497-2743Taichi Shimazuhttps://orcid.org/0000-0001-6000-9830Masamitsu Kamadahttps://orcid.org/0000-0003-1703-076XNorito Kawakamihttps://orcid.org/0000-0003-1080-2720Daisuke Nishihttps://orcid.org/0000-0001-9349-3294 BackgroundDigital mental health interventions are being used more than ever for the prevention and treatment of psychological problems. Optimizing the implementation aspects of digital mental health is essential to deliver the program to populations in need, but there is a lack of validated implementation outcome measures for digital mental health interventions. ObjectiveThe primary aim of this study is to develop implementation outcome scales of digital mental health for different levels of stakeholders involved in the implementation process: users, providers, and managers or policy makers. The secondary aim is to validate the developed scale for users. MethodsWe developed English and Japanese versions of the implementation outcome scales for digital mental health (iOSDMH) based on the literature review and panel discussions with experts in implementation research and web-based psychotherapy. The study developed acceptability, appropriateness, feasibility, satisfaction, and harm as the outcome measures for users, providers, and managers or policy makers. We conducted evidence-based interventions via the internet using UTSMeD, a website for mental health information (N=200). Exploratory factor analysis (EFA) was conducted to assess the structural validity of the iOSDMH for users. Satisfaction, which consisted of a single item, was not included in the EFA. ResultsThe iOSDMH was developed for users, providers, and managers or policy makers. The iOSDMH contains 19 items for users, 11 items for providers, and 14 items for managers or policy makers. Cronbach α coefficients indicated intermediate internal consistency for acceptability (α=.665) but high consistency for appropriateness (α=.776), feasibility (α=.832), and harm (α=.777) of the iOSDMH for users. EFA revealed 3-factor structures, indicating acceptability and appropriateness as close concepts. Despite the similarity between these 2 concepts, we inferred that acceptability and appropriateness should be used as different factors, following previous studies. ConclusionsWe developed iOSDMH for users, providers, and managers. Psychometric assessment of the scales for users demonstrated acceptable reliability and validity. Evaluating the components of digital mental health implementation is a major step forward in implementation science.https://formative.jmir.org/2021/11/e24332 |
| spellingShingle | Natsu Sasaki Erika Obikane Rajesh Vedanthan Kotaro Imamura Pim Cuijpers Taichi Shimazu Masamitsu Kamada Norito Kawakami Daisuke Nishi Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study JMIR Formative Research |
| title | Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study |
| title_full | Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study |
| title_fullStr | Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study |
| title_full_unstemmed | Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study |
| title_short | Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study |
| title_sort | implementation outcome scales for digital mental health iosdmh scale development and cross sectional study |
| url | https://formative.jmir.org/2021/11/e24332 |
| work_keys_str_mv | AT natsusasaki implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT erikaobikane implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT rajeshvedanthan implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT kotaroimamura implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT pimcuijpers implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT taichishimazu implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT masamitsukamada implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT noritokawakami implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy AT daisukenishi implementationoutcomescalesfordigitalmentalhealthiosdmhscaledevelopmentandcrosssectionalstudy |