Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning
This paper presents a phonological feature based computer aided pronunciation training system for the learners of a new language (L2). Phonological features allow analysing the learners’ mispronunciations systematically and rendering the feedback more effectively. The proposed acoustic model consist...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Published: |
International Speech Communication Association
2017
|
| _version_ | 1797073422158659584 |
|---|---|
| author | Arora, V Lahiri, A Reetz, H |
| author_facet | Arora, V Lahiri, A Reetz, H |
| author_sort | Arora, V |
| collection | OXFORD |
| description | This paper presents a phonological feature based computer aided pronunciation training system for the learners of a new language (L2). Phonological features allow analysing the learners’ mispronunciations systematically and rendering the feedback more effectively. The proposed acoustic model consists of a multi-task deep neural network, which uses a shared representation for estimating the phonological features and HMM state probabilities. Moreover, an active learning based scheme is proposed to efficiently deal with the cost of annotation, which is done by expert teachers, by selecting the most informative samples for annotation. Experimental evaluations are carried out for German and Italian native-speakers speaking English. For mispronunciation detection, the proposed feature-based system outperforms conventional GOP measure and classifier based methods, while providing more detailed diagnosis. Evaluations also demonstrate the advantage of active learning based sampling over random sampling. |
| first_indexed | 2024-03-06T23:21:54Z |
| format | Conference item |
| id | oxford-uuid:69032bc7-d9b0-45e6-b624-2822097a6f33 |
| institution | University of Oxford |
| last_indexed | 2024-03-06T23:21:54Z |
| publishDate | 2017 |
| publisher | International Speech Communication Association |
| record_format | dspace |
| spelling | oxford-uuid:69032bc7-d9b0-45e6-b624-2822097a6f332022-03-26T18:48:42ZPhonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active LearningConference itemhttp://purl.org/coar/resource_type/c_5794uuid:69032bc7-d9b0-45e6-b624-2822097a6f33Symplectic Elements at OxfordInternational Speech Communication Association2017Arora, VLahiri, AReetz, HThis paper presents a phonological feature based computer aided pronunciation training system for the learners of a new language (L2). Phonological features allow analysing the learners’ mispronunciations systematically and rendering the feedback more effectively. The proposed acoustic model consists of a multi-task deep neural network, which uses a shared representation for estimating the phonological features and HMM state probabilities. Moreover, an active learning based scheme is proposed to efficiently deal with the cost of annotation, which is done by expert teachers, by selecting the most informative samples for annotation. Experimental evaluations are carried out for German and Italian native-speakers speaking English. For mispronunciation detection, the proposed feature-based system outperforms conventional GOP measure and classifier based methods, while providing more detailed diagnosis. Evaluations also demonstrate the advantage of active learning based sampling over random sampling. |
| spellingShingle | Arora, V Lahiri, A Reetz, H Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning |
| title | Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning |
| title_full | Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning |
| title_fullStr | Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning |
| title_full_unstemmed | Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning |
| title_short | Phonological Feature Based Mispronunciation Detection and Diagnosis using Multi-Task DNNs and Active Learning |
| title_sort | phonological feature based mispronunciation detection and diagnosis using multi task dnns and active learning |
| work_keys_str_mv | AT arorav phonologicalfeaturebasedmispronunciationdetectionanddiagnosisusingmultitaskdnnsandactivelearning AT lahiria phonologicalfeaturebasedmispronunciationdetectionanddiagnosisusingmultitaskdnnsandactivelearning AT reetzh phonologicalfeaturebasedmispronunciationdetectionanddiagnosisusingmultitaskdnnsandactivelearning |