App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review
BackgroundDiagnostic delays in autism are common, with the time to diagnosis being up to 3 years from the onset of symptoms. Such delays have a proven detrimental effect on individuals and families going through the process. Digital health products, such as mobile apps, can h...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-11-01
|
| Series: | JMIR mHealth and uHealth |
| Online Access: | https://mhealth.jmir.org/2023/1/e52377 |
| _version_ | 1797608447485673472 |
|---|---|
| author | Sonia Ponzo Merle May Miren Tamayo-Elizalde Kerri Bailey Alanna J Shand Ryan Bamford Jan Multmeier Ivan Griessel Benedek Szulyovszky William Blakey Sophie Valentine David Plans |
| author_facet | Sonia Ponzo Merle May Miren Tamayo-Elizalde Kerri Bailey Alanna J Shand Ryan Bamford Jan Multmeier Ivan Griessel Benedek Szulyovszky William Blakey Sophie Valentine David Plans |
| author_sort | Sonia Ponzo |
| collection | DOAJ |
| description |
BackgroundDiagnostic delays in autism are common, with the time to diagnosis being up to 3 years from the onset of symptoms. Such delays have a proven detrimental effect on individuals and families going through the process. Digital health products, such as mobile apps, can help close this gap due to their scalability and ease of access. Further, mobile apps offer the opportunity to make the diagnostic process faster and more accurate by providing additional and timely information to clinicians undergoing autism assessments.
ObjectiveThe aim of this scoping review was to synthesize the available evidence about digital biomarker tools to aid clinicians, researchers in the autism field, and end users in making decisions as to their adoption within clinical and research settings.
MethodsWe conducted a structured literature search on databases and search engines to identify peer-reviewed studies and regulatory submissions that describe app characteristics, validation study details, and accuracy and validity metrics of commercial and research digital biomarker apps aimed at aiding the diagnosis of autism.
ResultsWe identified 4 studies evaluating 4 products: 1 commercial and 3 research apps. The accuracy of the identified apps varied between 28% and 80.6%. Sensitivity and specificity also varied, ranging from 51.6% to 81.6% and 18.5% to 80.5%, respectively. Positive predictive value ranged from 20.3% to 76.6%, and negative predictive value fluctuated between 48.7% and 97.4%. Further, we found a lack of details around participants’ demographics and, where these were reported, important imbalances in sex and ethnicity in the studies evaluating such products. Finally, evaluation methods as well as accuracy and validity metrics of available tools were not clearly reported in some cases and varied greatly across studies. Different comparators were also used, with some studies validating their tools against the Diagnostic and Statistical Manual of Mental Disorders criteria and others through self-reported measures. Further, while in most cases, 2 classes were used for algorithm validation purposes, 1 of the studies reported a third category (indeterminate). These discrepancies substantially impact the comparability and generalizability of the results, thus highlighting the need for standardized validation processes and the reporting of findings.
ConclusionsDespite their popularity, systematic evaluations and syntheses of the current state of the art of digital health products are lacking. Standardized and transparent evaluations of digital health tools in diverse populations are needed to assess their real-world usability and validity, as well as help researchers, clinicians, and end users safely adopt novel tools within clinical and research practices. |
| first_indexed | 2024-03-11T05:44:32Z |
| format | Article |
| id | doaj.art-242d4cf0359949619636f8fac8bb5a49 |
| institution | Directory Open Access Journal |
| issn | 2291-5222 |
| language | English |
| last_indexed | 2024-03-11T05:44:32Z |
| publishDate | 2023-11-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR mHealth and uHealth |
| spelling | doaj.art-242d4cf0359949619636f8fac8bb5a492023-11-17T15:00:42ZengJMIR PublicationsJMIR mHealth and uHealth2291-52222023-11-0111e5237710.2196/52377App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping ReviewSonia Ponzohttps://orcid.org/0000-0002-6754-5078Merle Mayhttps://orcid.org/0000-0002-6169-3934Miren Tamayo-Elizaldehttps://orcid.org/0000-0002-2769-0019Kerri Baileyhttps://orcid.org/0000-0002-8186-9730Alanna J Shandhttps://orcid.org/0009-0000-2540-6320Ryan Bamfordhttps://orcid.org/0000-0002-1229-3782Jan Multmeierhttps://orcid.org/0000-0002-2825-6716Ivan Griesselhttps://orcid.org/0009-0009-7885-3031Benedek Szulyovszkyhttps://orcid.org/0009-0004-0159-0206William Blakeyhttps://orcid.org/0009-0002-7550-3182Sophie Valentinehttps://orcid.org/0000-0002-4016-5075David Planshttps://orcid.org/0000-0002-0476-3342 BackgroundDiagnostic delays in autism are common, with the time to diagnosis being up to 3 years from the onset of symptoms. Such delays have a proven detrimental effect on individuals and families going through the process. Digital health products, such as mobile apps, can help close this gap due to their scalability and ease of access. Further, mobile apps offer the opportunity to make the diagnostic process faster and more accurate by providing additional and timely information to clinicians undergoing autism assessments. ObjectiveThe aim of this scoping review was to synthesize the available evidence about digital biomarker tools to aid clinicians, researchers in the autism field, and end users in making decisions as to their adoption within clinical and research settings. MethodsWe conducted a structured literature search on databases and search engines to identify peer-reviewed studies and regulatory submissions that describe app characteristics, validation study details, and accuracy and validity metrics of commercial and research digital biomarker apps aimed at aiding the diagnosis of autism. ResultsWe identified 4 studies evaluating 4 products: 1 commercial and 3 research apps. The accuracy of the identified apps varied between 28% and 80.6%. Sensitivity and specificity also varied, ranging from 51.6% to 81.6% and 18.5% to 80.5%, respectively. Positive predictive value ranged from 20.3% to 76.6%, and negative predictive value fluctuated between 48.7% and 97.4%. Further, we found a lack of details around participants’ demographics and, where these were reported, important imbalances in sex and ethnicity in the studies evaluating such products. Finally, evaluation methods as well as accuracy and validity metrics of available tools were not clearly reported in some cases and varied greatly across studies. Different comparators were also used, with some studies validating their tools against the Diagnostic and Statistical Manual of Mental Disorders criteria and others through self-reported measures. Further, while in most cases, 2 classes were used for algorithm validation purposes, 1 of the studies reported a third category (indeterminate). These discrepancies substantially impact the comparability and generalizability of the results, thus highlighting the need for standardized validation processes and the reporting of findings. ConclusionsDespite their popularity, systematic evaluations and syntheses of the current state of the art of digital health products are lacking. Standardized and transparent evaluations of digital health tools in diverse populations are needed to assess their real-world usability and validity, as well as help researchers, clinicians, and end users safely adopt novel tools within clinical and research practices.https://mhealth.jmir.org/2023/1/e52377 |
| spellingShingle | Sonia Ponzo Merle May Miren Tamayo-Elizalde Kerri Bailey Alanna J Shand Ryan Bamford Jan Multmeier Ivan Griessel Benedek Szulyovszky William Blakey Sophie Valentine David Plans App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review JMIR mHealth and uHealth |
| title | App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review |
| title_full | App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review |
| title_fullStr | App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review |
| title_full_unstemmed | App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review |
| title_short | App Characteristics and Accuracy Metrics of Available Digital Biomarkers for Autism: Scoping Review |
| title_sort | app characteristics and accuracy metrics of available digital biomarkers for autism scoping review |
| url | https://mhealth.jmir.org/2023/1/e52377 |
| work_keys_str_mv | AT soniaponzo appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT merlemay appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT mirentamayoelizalde appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT kerribailey appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT alannajshand appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT ryanbamford appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT janmultmeier appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT ivangriessel appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT benedekszulyovszky appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT williamblakey appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT sophievalentine appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview AT davidplans appcharacteristicsandaccuracymetricsofavailabledigitalbiomarkersforautismscopingreview |