Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory
In recent years, a lot of researchers’ attentions were concentrating on imaginary speech understanding, decoding, and even recognition. Speech is a complex mechanism, which involves multiple brain areas in the process of production, planning, and precise control of a large number of muscles and arti...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/22/11873 |
| _version_ | 1827644969718906880 |
|---|---|
| author | Ana-Luiza Rusnac Ovidiu Grigore |
| author_facet | Ana-Luiza Rusnac Ovidiu Grigore |
| author_sort | Ana-Luiza Rusnac |
| collection | DOAJ |
| description | In recent years, a lot of researchers’ attentions were concentrating on imaginary speech understanding, decoding, and even recognition. Speech is a complex mechanism, which involves multiple brain areas in the process of production, planning, and precise control of a large number of muscles and articulation involved in the actual utterance. This paper proposes an intelligent imaginary speech recognition system of eleven different utterances, seven phonemes, and four words from the Kara One database. We showed, during our research, that the feature space of the cross-covariance in frequency domain offers a better perspective of the imaginary speech by computing LDA for 2D representation of the feature space, in comparison to cross-covariance in the time domain and the raw signals without any processing. In the classification stage, we used a CNNLSTM neural network and obtained a performance of 43% accuracy for all eleven different utterances. The developed system was meant to be a subject’s shared system. We also showed that, using the channels corresponding to the anatomical structures of the brain involved in speech production, i.e., Broca area, primary motor cortex, and secondary motor cortex, 93% of information is preserved, obtaining 40% accuracy by using 29 electrodes out of the initial 62. |
| first_indexed | 2024-03-09T18:28:56Z |
| format | Article |
| id | doaj.art-06818926b68b43acb6c3718904e53575 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T18:28:56Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-06818926b68b43acb6c3718904e535752023-11-24T07:42:18ZengMDPI AGApplied Sciences2076-34172022-11-0112221187310.3390/app122211873Imaginary Speech Recognition Using a Convolutional Network with Long-Short MemoryAna-Luiza Rusnac0Ovidiu Grigore1Department of Applied Electronics and Information Engineering, Faculty of Electronics, Telecommunications and Information Technology, Polytechnic University of Bucharest, 060042 Bucharest, RomaniaDepartment of Applied Electronics and Information Engineering, Faculty of Electronics, Telecommunications and Information Technology, Polytechnic University of Bucharest, 060042 Bucharest, RomaniaIn recent years, a lot of researchers’ attentions were concentrating on imaginary speech understanding, decoding, and even recognition. Speech is a complex mechanism, which involves multiple brain areas in the process of production, planning, and precise control of a large number of muscles and articulation involved in the actual utterance. This paper proposes an intelligent imaginary speech recognition system of eleven different utterances, seven phonemes, and four words from the Kara One database. We showed, during our research, that the feature space of the cross-covariance in frequency domain offers a better perspective of the imaginary speech by computing LDA for 2D representation of the feature space, in comparison to cross-covariance in the time domain and the raw signals without any processing. In the classification stage, we used a CNNLSTM neural network and obtained a performance of 43% accuracy for all eleven different utterances. The developed system was meant to be a subject’s shared system. We also showed that, using the channels corresponding to the anatomical structures of the brain involved in speech production, i.e., Broca area, primary motor cortex, and secondary motor cortex, 93% of information is preserved, obtaining 40% accuracy by using 29 electrodes out of the initial 62.https://www.mdpi.com/2076-3417/12/22/11873imaginary speech recognitionelectroencephalographylong-short term memory neural networkconvolutional neural networkcross-covariance featuresspeech brain areas |
| spellingShingle | Ana-Luiza Rusnac Ovidiu Grigore Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory Applied Sciences imaginary speech recognition electroencephalography long-short term memory neural network convolutional neural network cross-covariance features speech brain areas |
| title | Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory |
| title_full | Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory |
| title_fullStr | Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory |
| title_full_unstemmed | Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory |
| title_short | Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory |
| title_sort | imaginary speech recognition using a convolutional network with long short memory |
| topic | imaginary speech recognition electroencephalography long-short term memory neural network convolutional neural network cross-covariance features speech brain areas |
| url | https://www.mdpi.com/2076-3417/12/22/11873 |
| work_keys_str_mv | AT analuizarusnac imaginaryspeechrecognitionusingaconvolutionalnetworkwithlongshortmemory AT ovidiugrigore imaginaryspeechrecognitionusingaconvolutionalnetworkwithlongshortmemory |