End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model
In this paper, we propose an end-to-end (E2E) neural network model to detect autism spectrum disorder (ASD) from children’s voices without explicitly extracting the deterministic features. In order to obtain the decisions for discriminating between the voices of children with ASD and those with typi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/1/202 |
| _version_ | 1797439709684695040 |
|---|---|
| author | Jung Hyuk Lee Geon Woo Lee Guiyoung Bong Hee Jeong Yoo Hong Kook Kim |
| author_facet | Jung Hyuk Lee Geon Woo Lee Guiyoung Bong Hee Jeong Yoo Hong Kook Kim |
| author_sort | Jung Hyuk Lee |
| collection | DOAJ |
| description | In this paper, we propose an end-to-end (E2E) neural network model to detect autism spectrum disorder (ASD) from children’s voices without explicitly extracting the deterministic features. In order to obtain the decisions for discriminating between the voices of children with ASD and those with typical development (TD), we combined two different feature-extraction models and a bidirectional long short-term memory (BLSTM)-based classifier to obtain the ASD/TD classification in the form of probability. We realized one of the feature extractors as the bottleneck feature from an autoencoder using the extended version of the Geneva minimalistic acoustic parameter set (eGeMAPS) input. The other feature extractor is the context vector from a pretrained wav2vec2.0-based model directly applied to the waveform input. In addition, we optimized the E2E models in two different ways: (1) fine-tuning and (2) joint optimization. To evaluate the performance of the proposed E2E models, we prepared two datasets from video recordings of ASD diagnoses collected between 2016 and 2018 at Seoul National University Bundang Hospital (SNUBH), and between 2019 and 2021 at a Living Lab. According to the experimental results, the proposed wav2vec2.0-based E2E model with joint optimization achieved significant improvements in the accuracy and unweighted average recall, from 64.74% to 71.66% and from 65.04% to 70.81%, respectively, compared with a conventional model using autoencoder-based BLSTM and the deterministic features of the eGeMAPS. |
| first_indexed | 2024-03-09T11:57:01Z |
| format | Article |
| id | doaj.art-e24dd8ce33574483861e7f9e4396355e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T11:57:01Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-e24dd8ce33574483861e7f9e4396355e2023-11-30T23:08:07ZengMDPI AGSensors1424-82202022-12-0123120210.3390/s23010202End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained ModelJung Hyuk Lee0Geon Woo Lee1Guiyoung Bong2Hee Jeong Yoo3Hong Kook Kim4School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of KoreaAI Graduate School, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of KoreaDepartment of Psychiatry, Seoul National University Bundang Hospital, Seongnam 13620, Republic of KoreaDepartment of Psychiatry, Seoul National University Bundang Hospital, Seongnam 13620, Republic of KoreaSchool of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of KoreaIn this paper, we propose an end-to-end (E2E) neural network model to detect autism spectrum disorder (ASD) from children’s voices without explicitly extracting the deterministic features. In order to obtain the decisions for discriminating between the voices of children with ASD and those with typical development (TD), we combined two different feature-extraction models and a bidirectional long short-term memory (BLSTM)-based classifier to obtain the ASD/TD classification in the form of probability. We realized one of the feature extractors as the bottleneck feature from an autoencoder using the extended version of the Geneva minimalistic acoustic parameter set (eGeMAPS) input. The other feature extractor is the context vector from a pretrained wav2vec2.0-based model directly applied to the waveform input. In addition, we optimized the E2E models in two different ways: (1) fine-tuning and (2) joint optimization. To evaluate the performance of the proposed E2E models, we prepared two datasets from video recordings of ASD diagnoses collected between 2016 and 2018 at Seoul National University Bundang Hospital (SNUBH), and between 2019 and 2021 at a Living Lab. According to the experimental results, the proposed wav2vec2.0-based E2E model with joint optimization achieved significant improvements in the accuracy and unweighted average recall, from 64.74% to 71.66% and from 65.04% to 70.81%, respectively, compared with a conventional model using autoencoder-based BLSTM and the deterministic features of the eGeMAPS.https://www.mdpi.com/1424-8220/23/1/202autism spectrum disorderend-to-end neural networkpretrained modeljoint optimizationautoencoderbidirectional long short-term memory (BLSTM) |
| spellingShingle | Jung Hyuk Lee Geon Woo Lee Guiyoung Bong Hee Jeong Yoo Hong Kook Kim End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model Sensors autism spectrum disorder end-to-end neural network pretrained model joint optimization autoencoder bidirectional long short-term memory (BLSTM) |
| title | End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model |
| title_full | End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model |
| title_fullStr | End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model |
| title_full_unstemmed | End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model |
| title_short | End-to-End Model-Based Detection of Infants with Autism Spectrum Disorder Using a Pretrained Model |
| title_sort | end to end model based detection of infants with autism spectrum disorder using a pretrained model |
| topic | autism spectrum disorder end-to-end neural network pretrained model joint optimization autoencoder bidirectional long short-term memory (BLSTM) |
| url | https://www.mdpi.com/1424-8220/23/1/202 |
| work_keys_str_mv | AT junghyuklee endtoendmodelbaseddetectionofinfantswithautismspectrumdisorderusingapretrainedmodel AT geonwoolee endtoendmodelbaseddetectionofinfantswithautismspectrumdisorderusingapretrainedmodel AT guiyoungbong endtoendmodelbaseddetectionofinfantswithautismspectrumdisorderusingapretrainedmodel AT heejeongyoo endtoendmodelbaseddetectionofinfantswithautismspectrumdisorderusingapretrainedmodel AT hongkookkim endtoendmodelbaseddetectionofinfantswithautismspectrumdisorderusingapretrainedmodel |