Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods
BackgroundAutism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting time...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2022-02-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2022/2/e31830 |
| _version_ | 1797735342585937920 |
|---|---|
| author | Maya Varma Peter Washington Brianna Chrisman Aaron Kline Emilie Leblanc Kelley Paskov Nate Stockham Jae-Yoon Jung Min Woo Sun Dennis P Wall |
| author_facet | Maya Varma Peter Washington Brianna Chrisman Aaron Kline Emilie Leblanc Kelley Paskov Nate Stockham Jae-Yoon Jung Min Woo Sun Dennis P Wall |
| author_sort | Maya Varma |
| collection | DOAJ |
| description |
BackgroundAutism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process.
ObjectiveIn this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments.
MethodsHere, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual’s visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD.
ResultsOur results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD.
ConclusionsUltimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data. |
| first_indexed | 2024-03-12T12:57:38Z |
| format | Article |
| id | doaj.art-851fa73415b645c2a6e20c83b4c092bf |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T12:57:38Z |
| publishDate | 2022-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-851fa73415b645c2a6e20c83b4c092bf2023-08-28T20:48:08ZengJMIR PublicationsJournal of Medical Internet Research1438-88712022-02-01242e3183010.2196/31830Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning MethodsMaya Varmahttps://orcid.org/0000-0003-0693-7753Peter Washingtonhttps://orcid.org/0000-0003-3276-4411Brianna Chrismanhttps://orcid.org/0000-0002-7157-607XAaron Klinehttps://orcid.org/0000-0002-0077-5485Emilie Leblanchttps://orcid.org/0000-0002-3492-3554Kelley Paskovhttps://orcid.org/0000-0002-5252-1401Nate Stockhamhttps://orcid.org/0000-0002-0752-6801Jae-Yoon Junghttps://orcid.org/0000-0001-7948-9803Min Woo Sunhttps://orcid.org/0000-0003-1049-1854Dennis P Wallhttps://orcid.org/0000-0002-7889-9146 BackgroundAutism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process. ObjectiveIn this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments. MethodsHere, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual’s visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD. ResultsOur results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD. ConclusionsUltimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.https://www.jmir.org/2022/2/e31830 |
| spellingShingle | Maya Varma Peter Washington Brianna Chrisman Aaron Kline Emilie Leblanc Kelley Paskov Nate Stockham Jae-Yoon Jung Min Woo Sun Dennis P Wall Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods Journal of Medical Internet Research |
| title | Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods |
| title_full | Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods |
| title_fullStr | Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods |
| title_full_unstemmed | Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods |
| title_short | Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods |
| title_sort | identification of social engagement indicators associated with autism spectrum disorder using a game based mobile app comparative study of gaze fixation and visual scanning methods |
| url | https://www.jmir.org/2022/2/e31830 |
| work_keys_str_mv | AT mayavarma identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT peterwashington identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT briannachrisman identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT aaronkline identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT emilieleblanc identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT kelleypaskov identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT natestockham identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT jaeyoonjung identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT minwoosun identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods AT dennispwall identificationofsocialengagementindicatorsassociatedwithautismspectrumdisorderusingagamebasedmobileappcomparativestudyofgazefixationandvisualscanningmethods |