Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies
Aphasia is a type of speech disorder that can cause speech defects in a person. Identifying the severity level of the aphasia patient is critical for the rehabilitation process. In this research, we identify ten aphasia severity levels motivated by specific speech therapies based on the presence or...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/18/6966 |
| _version_ | 1827656766428545024 |
|---|---|
| author | Herath Mudiyanselage Dhammike Piyumal Madhurajith Herath Weraniyagoda Arachchilage Sahanaka Anuththara Weraniyagoda Rajapakshage Thilina Madhushan Rajapaksha Patikiri Arachchige Don Shehan Nilmantha Wijesekara Kalupahana Liyanage Kushan Sudheera Peter Han Joo Chong |
| author_facet | Herath Mudiyanselage Dhammike Piyumal Madhurajith Herath Weraniyagoda Arachchilage Sahanaka Anuththara Weraniyagoda Rajapakshage Thilina Madhushan Rajapaksha Patikiri Arachchige Don Shehan Nilmantha Wijesekara Kalupahana Liyanage Kushan Sudheera Peter Han Joo Chong |
| author_sort | Herath Mudiyanselage Dhammike Piyumal Madhurajith Herath |
| collection | DOAJ |
| description | Aphasia is a type of speech disorder that can cause speech defects in a person. Identifying the severity level of the aphasia patient is critical for the rehabilitation process. In this research, we identify ten aphasia severity levels motivated by specific speech therapies based on the presence or absence of identified characteristics in aphasic speech in order to give more specific treatment to the patient. In the aphasia severity level classification process, we experiment on different speech feature extraction techniques, lengths of input audio samples, and machine learning classifiers toward classification performance. Aphasic speech is required to be sensed by an audio sensor and then recorded and divided into audio frames and passed through an audio feature extractor before feeding into the machine learning classifier. According to the results, the mel frequency cepstral coefficient (MFCC) is the most suitable audio feature extraction method for the aphasic speech level classification process, as it outperformed the classification performance of all mel-spectrogram, chroma, and zero crossing rates by a large margin. Furthermore, the classification performance is higher when 20 s audio samples are used compared with 10 s chunks, even though the performance gap is narrow. Finally, the deep neural network approach resulted in the best classification performance, which was slightly better than both K-nearest neighbor (KNN) and random forest classifiers, and it was significantly better than decision tree algorithms. Therefore, the study shows that aphasia level classification can be completed with accuracy, precision, recall, and F1-score values of 0.99 using MFCC for 20 s audio samples using the deep neural network approach in order to recommend corresponding speech therapy for the identified level. A web application was developed for English-speaking aphasia patients to self-diagnose the severity level and engage in speech therapies. |
| first_indexed | 2024-03-09T22:33:55Z |
| format | Article |
| id | doaj.art-82a9e4d26fbd4a27b71b4171e0024515 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T22:33:55Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-82a9e4d26fbd4a27b71b4171e00245152023-11-23T18:52:17ZengMDPI AGSensors1424-82202022-09-012218696610.3390/s22186966Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech TherapiesHerath Mudiyanselage Dhammike Piyumal Madhurajith Herath0Weraniyagoda Arachchilage Sahanaka Anuththara Weraniyagoda1Rajapakshage Thilina Madhushan Rajapaksha2Patikiri Arachchige Don Shehan Nilmantha Wijesekara3Kalupahana Liyanage Kushan Sudheera4Peter Han Joo Chong5Department of Electrical and Information Engineering, Faculty of Engineering, University of Ruhuna, Galle 80000, Sri LankaDepartment of Electrical and Information Engineering, Faculty of Engineering, University of Ruhuna, Galle 80000, Sri LankaDepartment of Electrical and Information Engineering, Faculty of Engineering, University of Ruhuna, Galle 80000, Sri LankaDepartment of Electrical and Information Engineering, Faculty of Engineering, University of Ruhuna, Galle 80000, Sri LankaDepartment of Electrical and Information Engineering, Faculty of Engineering, University of Ruhuna, Galle 80000, Sri LankaDepartment of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New ZealandAphasia is a type of speech disorder that can cause speech defects in a person. Identifying the severity level of the aphasia patient is critical for the rehabilitation process. In this research, we identify ten aphasia severity levels motivated by specific speech therapies based on the presence or absence of identified characteristics in aphasic speech in order to give more specific treatment to the patient. In the aphasia severity level classification process, we experiment on different speech feature extraction techniques, lengths of input audio samples, and machine learning classifiers toward classification performance. Aphasic speech is required to be sensed by an audio sensor and then recorded and divided into audio frames and passed through an audio feature extractor before feeding into the machine learning classifier. According to the results, the mel frequency cepstral coefficient (MFCC) is the most suitable audio feature extraction method for the aphasic speech level classification process, as it outperformed the classification performance of all mel-spectrogram, chroma, and zero crossing rates by a large margin. Furthermore, the classification performance is higher when 20 s audio samples are used compared with 10 s chunks, even though the performance gap is narrow. Finally, the deep neural network approach resulted in the best classification performance, which was slightly better than both K-nearest neighbor (KNN) and random forest classifiers, and it was significantly better than decision tree algorithms. Therefore, the study shows that aphasia level classification can be completed with accuracy, precision, recall, and F1-score values of 0.99 using MFCC for 20 s audio samples using the deep neural network approach in order to recommend corresponding speech therapy for the identified level. A web application was developed for English-speaking aphasia patients to self-diagnose the severity level and engage in speech therapies.https://www.mdpi.com/1424-8220/22/18/6966aphasiaseverity leveldeep neural networksmel frequency cepstral coefficientsaudio sensorsspeech therapy |
| spellingShingle | Herath Mudiyanselage Dhammike Piyumal Madhurajith Herath Weraniyagoda Arachchilage Sahanaka Anuththara Weraniyagoda Rajapakshage Thilina Madhushan Rajapaksha Patikiri Arachchige Don Shehan Nilmantha Wijesekara Kalupahana Liyanage Kushan Sudheera Peter Han Joo Chong Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies Sensors aphasia severity level deep neural networks mel frequency cepstral coefficients audio sensors speech therapy |
| title | Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies |
| title_full | Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies |
| title_fullStr | Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies |
| title_full_unstemmed | Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies |
| title_short | Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies |
| title_sort | automatic assessment of aphasic speech sensed by audio sensors for classification into aphasia severity levels to recommend speech therapies |
| topic | aphasia severity level deep neural networks mel frequency cepstral coefficients audio sensors speech therapy |
| url | https://www.mdpi.com/1424-8220/22/18/6966 |
| work_keys_str_mv | AT herathmudiyanselagedhammikepiyumalmadhurajithherath automaticassessmentofaphasicspeechsensedbyaudiosensorsforclassificationintoaphasiaseveritylevelstorecommendspeechtherapies AT weraniyagodaarachchilagesahanakaanuththaraweraniyagoda automaticassessmentofaphasicspeechsensedbyaudiosensorsforclassificationintoaphasiaseveritylevelstorecommendspeechtherapies AT rajapakshagethilinamadhushanrajapaksha automaticassessmentofaphasicspeechsensedbyaudiosensorsforclassificationintoaphasiaseveritylevelstorecommendspeechtherapies AT patikiriarachchigedonshehannilmanthawijesekara automaticassessmentofaphasicspeechsensedbyaudiosensorsforclassificationintoaphasiaseveritylevelstorecommendspeechtherapies AT kalupahanaliyanagekushansudheera automaticassessmentofaphasicspeechsensedbyaudiosensorsforclassificationintoaphasiaseveritylevelstorecommendspeechtherapies AT peterhanjoochong automaticassessmentofaphasicspeechsensedbyaudiosensorsforclassificationintoaphasiaseveritylevelstorecommendspeechtherapies |