In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV
Most conventional speech recognition systems have mainly concentrated on voice-driven control of personal user devices such as smartphones. Therefore, a speech recognition system used in a special environment needs to be developed in consideration of the environment. In this study, a speech recognit...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Aerospace |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2226-4310/10/10/841 |
| _version_ | 1797575076356292608 |
|---|---|
| author | Jeong-Sik Park Na Geng |
| author_facet | Jeong-Sik Park Na Geng |
| author_sort | Jeong-Sik Park |
| collection | DOAJ |
| description | Most conventional speech recognition systems have mainly concentrated on voice-driven control of personal user devices such as smartphones. Therefore, a speech recognition system used in a special environment needs to be developed in consideration of the environment. In this study, a speech recognition framework for voice-driven control of unmanned aerial vehicles (UAVs) is proposed in a collaborative environment between manned aerial vehicles (MAVs) and UAVs, where multiple MAVs and UAVs fly together, and pilots on board MAVs control multiple UAVs with their voices. Standard speech recognition systems consist of several modules, including front-end, recognition, and post-processing. Among them, this study focuses on recognition and post-processing modules in terms of in-vehicle speech recognition. In order to stably control UAVs via voice, it is necessary to handle the environmental conditions of the UAVs carefully. First, we define control commands that the MAV pilot delivers to UAVs and construct training data. Next, for the recognition module, we investigate an acoustic model suitable for the characteristics of the UAV control commands and the UAV system with hardware resource constraints. Finally, two approaches are proposed for post-processing: grammar network-based syntax analysis and transaction-based semantic analysis. For evaluation, we developed a speech recognition system in a collaborative simulation environment between a MAV and an UAV and successfully verified the validity of each module. As a result of recognition experiments of connected words consisting of two to five words, the recognition rates of hidden Markov model (HMM) and deep neural network (DNN)-based acoustic models were 98.2% and 98.4%, respectively. However, in terms of computational amount, the HMM model was about 100 times more efficient than DNN. In addition, the relative improvement in error rate with the proposed post-processing was about 65%. |
| first_indexed | 2024-03-10T21:31:11Z |
| format | Article |
| id | doaj.art-51dfef8b9b0148e38f3c1578bcb66c1f |
| institution | Directory Open Access Journal |
| issn | 2226-4310 |
| language | English |
| last_indexed | 2024-03-10T21:31:11Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Aerospace |
| spelling | doaj.art-51dfef8b9b0148e38f3c1578bcb66c1f2023-11-19T15:16:54ZengMDPI AGAerospace2226-43102023-09-01101084110.3390/aerospace10100841In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAVJeong-Sik Park0Na Geng1Department of English Linguistics and Language Technology, Hankuk University of Foreign Studies, Seoul 02450, Republic of KoreaDepartment of English Linguistics, Hankuk University of Foreign Studies, Seoul 02450, Republic of KoreaMost conventional speech recognition systems have mainly concentrated on voice-driven control of personal user devices such as smartphones. Therefore, a speech recognition system used in a special environment needs to be developed in consideration of the environment. In this study, a speech recognition framework for voice-driven control of unmanned aerial vehicles (UAVs) is proposed in a collaborative environment between manned aerial vehicles (MAVs) and UAVs, where multiple MAVs and UAVs fly together, and pilots on board MAVs control multiple UAVs with their voices. Standard speech recognition systems consist of several modules, including front-end, recognition, and post-processing. Among them, this study focuses on recognition and post-processing modules in terms of in-vehicle speech recognition. In order to stably control UAVs via voice, it is necessary to handle the environmental conditions of the UAVs carefully. First, we define control commands that the MAV pilot delivers to UAVs and construct training data. Next, for the recognition module, we investigate an acoustic model suitable for the characteristics of the UAV control commands and the UAV system with hardware resource constraints. Finally, two approaches are proposed for post-processing: grammar network-based syntax analysis and transaction-based semantic analysis. For evaluation, we developed a speech recognition system in a collaborative simulation environment between a MAV and an UAV and successfully verified the validity of each module. As a result of recognition experiments of connected words consisting of two to five words, the recognition rates of hidden Markov model (HMM) and deep neural network (DNN)-based acoustic models were 98.2% and 98.4%, respectively. However, in terms of computational amount, the HMM model was about 100 times more efficient than DNN. In addition, the relative improvement in error rate with the proposed post-processing was about 65%.https://www.mdpi.com/2226-4310/10/10/841speech recognitionvoice-driven controlacoustic modelgrammar networksyntax analysissemantic analysis |
| spellingShingle | Jeong-Sik Park Na Geng In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV Aerospace speech recognition voice-driven control acoustic model grammar network syntax analysis semantic analysis |
| title | In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV |
| title_full | In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV |
| title_fullStr | In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV |
| title_full_unstemmed | In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV |
| title_short | In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV |
| title_sort | in vehicle speech recognition for voice driven uav control in a collaborative environment of mav and uav |
| topic | speech recognition voice-driven control acoustic model grammar network syntax analysis semantic analysis |
| url | https://www.mdpi.com/2226-4310/10/10/841 |
| work_keys_str_mv | AT jeongsikpark invehiclespeechrecognitionforvoicedrivenuavcontrolinacollaborativeenvironmentofmavanduav AT nageng invehiclespeechrecognitionforvoicedrivenuavcontrolinacollaborativeenvironmentofmavanduav |