Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA)
The Montreal cognitive assessment (MoCA), a widely accepted screening tool for identifying patients with mild cognitive impairment (MCI), includes a language fluency test of verbal functioning; its scores are based on the number of unique correct words produced by the test taker. However, it is poss...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/4/1583 |
| _version_ | 1797476692375109632 |
|---|---|
| author | Pimarn Kantithammakorn Proadpran Punyabukkana Ploy N. Pratanwanich Solaphat Hemrungrojn Chaipat Chunharas Dittaya Wanvarie |
| author_facet | Pimarn Kantithammakorn Proadpran Punyabukkana Ploy N. Pratanwanich Solaphat Hemrungrojn Chaipat Chunharas Dittaya Wanvarie |
| author_sort | Pimarn Kantithammakorn |
| collection | DOAJ |
| description | The Montreal cognitive assessment (MoCA), a widely accepted screening tool for identifying patients with mild cognitive impairment (MCI), includes a language fluency test of verbal functioning; its scores are based on the number of unique correct words produced by the test taker. However, it is possible that unique words may be counted differently for various languages. This study focuses on Thai as a language that differs from English in terms of word combinations. We applied various automatic speech recognition (ASR) techniques to develop an assisted scoring system for the MoCA language fluency test with Thai language support. This was a challenge because Thai is a low-resource language for which domain-specific data are not publicly available, especially speech data from patients with MCIs. Furthermore, the great variety of pronunciation, intonation, tone, and accent of the patients, all of which might differ from healthy controls, bring more complexity to the model. We propose a hybrid time delay neural network hidden Markov model (TDNN-HMM) architecture for acoustic model training to create our ASR system that is robust to environmental noise and to the variation of voice quality impacted by MCI. The LOTUS Thai speech corpus was incorporated into the training set to improve the model’s generalization. A preprocessing algorithm was implemented to reduce the background noise and improve the overall data quality before feeding data into the TDNN-HMM system for automatic word detection and language fluency score calculation. The results show that the TDNN-HMM model in combination with data augmentation using lattice-free maximum mutual information (LF-MMI) objective function provides a word error rate (WER) of 30.77%. To our knowledge, this is the first study to develop an ASR with Thai language support to automate the scoring system of MoCA’s language fluency assessment. |
| first_indexed | 2024-03-09T21:06:15Z |
| format | Article |
| id | doaj.art-7db91cff1b9f49528db1b077f30cad81 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T21:06:15Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-7db91cff1b9f49528db1b077f30cad812023-11-23T22:01:50ZengMDPI AGSensors1424-82202022-02-01224158310.3390/s22041583Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA)Pimarn Kantithammakorn0Proadpran Punyabukkana1Ploy N. Pratanwanich2Solaphat Hemrungrojn3Chaipat Chunharas4Dittaya Wanvarie5Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, ThailandDepartment of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, ThailandDepartment of Mathematics and Computer Science, Faculty of Science, Chulalongkorn University, Bangkok 10330, ThailandDepartment of Psychiatry, Faculty of Medicine, Chulalongkorn University, Bangkok 10330, ThailandCognitive Clinical & Computational Neuroscience Research Unit, Department of Internal Medicine, Faculty of Medicine, Chulalongkorn University, Bangkok 10330, ThailandDepartment of Mathematics and Computer Science, Faculty of Science, Chulalongkorn University, Bangkok 10330, ThailandThe Montreal cognitive assessment (MoCA), a widely accepted screening tool for identifying patients with mild cognitive impairment (MCI), includes a language fluency test of verbal functioning; its scores are based on the number of unique correct words produced by the test taker. However, it is possible that unique words may be counted differently for various languages. This study focuses on Thai as a language that differs from English in terms of word combinations. We applied various automatic speech recognition (ASR) techniques to develop an assisted scoring system for the MoCA language fluency test with Thai language support. This was a challenge because Thai is a low-resource language for which domain-specific data are not publicly available, especially speech data from patients with MCIs. Furthermore, the great variety of pronunciation, intonation, tone, and accent of the patients, all of which might differ from healthy controls, bring more complexity to the model. We propose a hybrid time delay neural network hidden Markov model (TDNN-HMM) architecture for acoustic model training to create our ASR system that is robust to environmental noise and to the variation of voice quality impacted by MCI. The LOTUS Thai speech corpus was incorporated into the training set to improve the model’s generalization. A preprocessing algorithm was implemented to reduce the background noise and improve the overall data quality before feeding data into the TDNN-HMM system for automatic word detection and language fluency score calculation. The results show that the TDNN-HMM model in combination with data augmentation using lattice-free maximum mutual information (LF-MMI) objective function provides a word error rate (WER) of 30.77%. To our knowledge, this is the first study to develop an ASR with Thai language support to automate the scoring system of MoCA’s language fluency assessment.https://www.mdpi.com/1424-8220/22/4/1583MoCAASRspeech recognitionscoringlanguage fluency testThai tonal language |
| spellingShingle | Pimarn Kantithammakorn Proadpran Punyabukkana Ploy N. Pratanwanich Solaphat Hemrungrojn Chaipat Chunharas Dittaya Wanvarie Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA) Sensors MoCA ASR speech recognition scoring language fluency test Thai tonal language |
| title | Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA) |
| title_full | Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA) |
| title_fullStr | Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA) |
| title_full_unstemmed | Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA) |
| title_short | Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA) |
| title_sort | using automatic speech recognition to assess thai speech language fluency in the montreal cognitive assessment moca |
| topic | MoCA ASR speech recognition scoring language fluency test Thai tonal language |
| url | https://www.mdpi.com/1424-8220/22/4/1583 |
| work_keys_str_mv | AT pimarnkantithammakorn usingautomaticspeechrecognitiontoassessthaispeechlanguagefluencyinthemontrealcognitiveassessmentmoca AT proadpranpunyabukkana usingautomaticspeechrecognitiontoassessthaispeechlanguagefluencyinthemontrealcognitiveassessmentmoca AT ploynpratanwanich usingautomaticspeechrecognitiontoassessthaispeechlanguagefluencyinthemontrealcognitiveassessmentmoca AT solaphathemrungrojn usingautomaticspeechrecognitiontoassessthaispeechlanguagefluencyinthemontrealcognitiveassessmentmoca AT chaipatchunharas usingautomaticspeechrecognitiontoassessthaispeechlanguagefluencyinthemontrealcognitiveassessmentmoca AT dittayawanvarie usingautomaticspeechrecognitiontoassessthaispeechlanguagefluencyinthemontrealcognitiveassessmentmoca |