Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognit...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/2/411 |
| _version_ | 1827603076540792832 |
|---|---|
| author | Yunkai Zhang Yinghong Tian Pingyi Wu Dongfan Chen |
| author_facet | Yunkai Zhang Yinghong Tian Pingyi Wu Dongfan Chen |
| author_sort | Yunkai Zhang |
| collection | DOAJ |
| description | The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children’s actions. The performed experiments show that our proposed method is effective for ASD children’s action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children. |
| first_indexed | 2024-03-09T05:32:34Z |
| format | Article |
| id | doaj.art-b5b9b3e5acab41ad8ccc4faf96fed9e5 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T05:32:34Z |
| publishDate | 2021-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-b5b9b3e5acab41ad8ccc4faf96fed9e52023-12-03T12:31:57ZengMDPI AGSensors1424-82202021-01-0121241110.3390/s21020411Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum DisorderYunkai Zhang0Yinghong Tian1Pingyi Wu2Dongfan Chen3School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, ChinaSchool of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, ChinaExperimental Teaching Center for Teacher Education, East China Normal University, Shanghai 200241, ChinaDepartment of Rehabilitation Sciences, East China Normal University, Shanghai 200062, ChinaThe recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children’s actions. The performed experiments show that our proposed method is effective for ASD children’s action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children.https://www.mdpi.com/1424-8220/21/2/411ASD childrenaction recognitionLSTMskeleton data |
| spellingShingle | Yunkai Zhang Yinghong Tian Pingyi Wu Dongfan Chen Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder Sensors ASD children action recognition LSTM skeleton data |
| title | Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder |
| title_full | Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder |
| title_fullStr | Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder |
| title_full_unstemmed | Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder |
| title_short | Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder |
| title_sort | application of skeleton data and long short term memory in action recognition of children with autism spectrum disorder |
| topic | ASD children action recognition LSTM skeleton data |
| url | https://www.mdpi.com/1424-8220/21/2/411 |
| work_keys_str_mv | AT yunkaizhang applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder AT yinghongtian applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder AT pingyiwu applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder AT dongfanchen applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder |