Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study
BackgroundThere are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards. ObjectiveThe objective of this paper was to examine...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-11-01
|
| Series: | JMIR mHealth and uHealth |
| Online Access: | https://mhealth.jmir.org/2023/1/e47043 |
| _version_ | 1797491854788263936 |
|---|---|
| author | Maciej Hyzy Raymond Bond Maurice Mulvenna Lu Bai Alan Dix Robert Daly Anna-Lena Frey Simon Leigh |
| author_facet | Maciej Hyzy Raymond Bond Maurice Mulvenna Lu Bai Alan Dix Robert Daly Anna-Lena Frey Simon Leigh |
| author_sort | Maciej Hyzy |
| collection | DOAJ |
| description |
BackgroundThere are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards.
ObjectiveThe objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP).
MethodsWe collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis.
ResultsWe found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI’s health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score.
ConclusionsThere is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women’s health, sexual health, and dental care may be needed. |
| first_indexed | 2024-03-10T00:55:14Z |
| format | Article |
| id | doaj.art-e56faa9f4d36403291713c87e2500e1c |
| institution | Directory Open Access Journal |
| issn | 2291-5222 |
| language | English |
| last_indexed | 2024-03-10T00:55:14Z |
| publishDate | 2023-11-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR mHealth and uHealth |
| spelling | doaj.art-e56faa9f4d36403291713c87e2500e1c2023-11-23T14:45:41ZengJMIR PublicationsJMIR mHealth and uHealth2291-52222023-11-0111e4704310.2196/47043Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis StudyMaciej Hyzyhttps://orcid.org/0000-0003-0791-8976Raymond Bondhttps://orcid.org/0000-0002-1078-2232Maurice Mulvennahttps://orcid.org/0000-0002-1554-0785Lu Baihttps://orcid.org/0000-0003-1242-5412Alan Dixhttps://orcid.org/0000-0002-5242-7693Robert Dalyhttps://orcid.org/0000-0001-7360-0558Anna-Lena Freyhttps://orcid.org/0000-0003-1653-3023Simon Leighhttps://orcid.org/0000-0002-6843-6447 BackgroundThere are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards. ObjectiveThe objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP). MethodsWe collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis. ResultsWe found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI’s health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score. ConclusionsThere is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women’s health, sexual health, and dental care may be needed.https://mhealth.jmir.org/2023/1/e47043 |
| spellingShingle | Maciej Hyzy Raymond Bond Maurice Mulvenna Lu Bai Alan Dix Robert Daly Anna-Lena Frey Simon Leigh Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study JMIR mHealth and uHealth |
| title | Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study |
| title_full | Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study |
| title_fullStr | Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study |
| title_full_unstemmed | Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study |
| title_short | Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study |
| title_sort | quality of digital health interventions across different health care domains secondary data analysis study |
| url | https://mhealth.jmir.org/2023/1/e47043 |
| work_keys_str_mv | AT maciejhyzy qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT raymondbond qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT mauricemulvenna qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT lubai qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT alandix qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT robertdaly qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT annalenafrey qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy AT simonleigh qualityofdigitalhealthinterventionsacrossdifferenthealthcaredomainssecondarydataanalysisstudy |