Real-time Controlling Dynamics Sensing in Air Traffic System
In order to obtain real-time controlling dynamics in air traffic system, a framework is proposed to introduce and process air traffic control (ATC) speech via radiotelephony communication. An automatic speech recognition (ASR) and controlling instruction understanding (CIU)-based pipeline is designe...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/3/679 |
| _version_ | 1798040407403134976 |
|---|---|
| author | Yi Lin Xianlong Tan Bo Yang Kai Yang Jianwei Zhang Jing Yu |
| author_facet | Yi Lin Xianlong Tan Bo Yang Kai Yang Jianwei Zhang Jing Yu |
| author_sort | Yi Lin |
| collection | DOAJ |
| description | In order to obtain real-time controlling dynamics in air traffic system, a framework is proposed to introduce and process air traffic control (ATC) speech via radiotelephony communication. An automatic speech recognition (ASR) and controlling instruction understanding (CIU)-based pipeline is designed to convert the ATC speech into ATC related elements, i.e., controlling intent and parameters. A correction procedure is also proposed to improve the reliability of the information obtained by the proposed framework. In the ASR model, acoustic model (AM), pronunciation model (PM), and phoneme- and word-based language model (LM) are proposed to unify multilingual ASR into one model. In this work, based on their tasks, the AM and PM are defined as speech recognition and machine translation problems respectively. Two-dimensional convolution and average-pooling layers are designed to solve special challenges of ASR in ATC. An encoder⁻decoder architecture-based neural network is proposed to translate phoneme labels into word labels, which achieves the purpose of ASR. In the CIU model, a recurrent neural network-based joint model is proposed to detect the controlling intent and label the controlling parameters, in which the two tasks are solved in one network to enhance the performance with each other based on ATC communication rules. The ATC speech is now converted into ATC related elements by the proposed ASR and CIU model. To further improve the accuracy of the sensing framework, a correction procedure is proposed to revise minor mistakes in ASR decoding results based on the flight information, such as flight plan, ADS-B. The proposed models are trained using real operating data and applied to a civil aviation airport in China to evaluate their performance. Experimental results show that the proposed framework can obtain real-time controlling dynamics with high performance, only 4% word-error rate. Meanwhile, the decoding efficiency can also meet the requirement of real-time applications, i.e., an average 0.147 real time factor. With the proposed framework and obtained traffic dynamics, current ATC applications can be accomplished with higher accuracy. In addition, the proposed ASR pipeline has high reusability, which allows us to apply it to other controlling scenes and languages with minor changes. |
| first_indexed | 2024-04-11T22:07:12Z |
| format | Article |
| id | doaj.art-84237fda40fc44bf9affa4514ce4215f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T22:07:12Z |
| publishDate | 2019-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-84237fda40fc44bf9affa4514ce4215f2022-12-22T04:00:40ZengMDPI AGSensors1424-82202019-02-0119367910.3390/s19030679s19030679Real-time Controlling Dynamics Sensing in Air Traffic SystemYi Lin0Xianlong Tan1Bo Yang2Kai Yang3Jianwei Zhang4Jing Yu5National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu 610065, ChinaSouthwest Air Traffic Management Bureau, Civil Aviation Administration of China, Chengdu 610000, ChinaNational Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu 610065, ChinaNational Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu 610065, ChinaNational Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu 610065, ChinaNational Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu 610065, ChinaIn order to obtain real-time controlling dynamics in air traffic system, a framework is proposed to introduce and process air traffic control (ATC) speech via radiotelephony communication. An automatic speech recognition (ASR) and controlling instruction understanding (CIU)-based pipeline is designed to convert the ATC speech into ATC related elements, i.e., controlling intent and parameters. A correction procedure is also proposed to improve the reliability of the information obtained by the proposed framework. In the ASR model, acoustic model (AM), pronunciation model (PM), and phoneme- and word-based language model (LM) are proposed to unify multilingual ASR into one model. In this work, based on their tasks, the AM and PM are defined as speech recognition and machine translation problems respectively. Two-dimensional convolution and average-pooling layers are designed to solve special challenges of ASR in ATC. An encoder⁻decoder architecture-based neural network is proposed to translate phoneme labels into word labels, which achieves the purpose of ASR. In the CIU model, a recurrent neural network-based joint model is proposed to detect the controlling intent and label the controlling parameters, in which the two tasks are solved in one network to enhance the performance with each other based on ATC communication rules. The ATC speech is now converted into ATC related elements by the proposed ASR and CIU model. To further improve the accuracy of the sensing framework, a correction procedure is proposed to revise minor mistakes in ASR decoding results based on the flight information, such as flight plan, ADS-B. The proposed models are trained using real operating data and applied to a civil aviation airport in China to evaluate their performance. Experimental results show that the proposed framework can obtain real-time controlling dynamics with high performance, only 4% word-error rate. Meanwhile, the decoding efficiency can also meet the requirement of real-time applications, i.e., an average 0.147 real time factor. With the proposed framework and obtained traffic dynamics, current ATC applications can be accomplished with higher accuracy. In addition, the proposed ASR pipeline has high reusability, which allows us to apply it to other controlling scenes and languages with minor changes.https://www.mdpi.com/1424-8220/19/3/679ATC speechautomatic speech recognitioncontrolling instruction understandingdeep learninglanguage modelaverage pooling |
| spellingShingle | Yi Lin Xianlong Tan Bo Yang Kai Yang Jianwei Zhang Jing Yu Real-time Controlling Dynamics Sensing in Air Traffic System Sensors ATC speech automatic speech recognition controlling instruction understanding deep learning language model average pooling |
| title | Real-time Controlling Dynamics Sensing in Air Traffic System |
| title_full | Real-time Controlling Dynamics Sensing in Air Traffic System |
| title_fullStr | Real-time Controlling Dynamics Sensing in Air Traffic System |
| title_full_unstemmed | Real-time Controlling Dynamics Sensing in Air Traffic System |
| title_short | Real-time Controlling Dynamics Sensing in Air Traffic System |
| title_sort | real time controlling dynamics sensing in air traffic system |
| topic | ATC speech automatic speech recognition controlling instruction understanding deep learning language model average pooling |
| url | https://www.mdpi.com/1424-8220/19/3/679 |
| work_keys_str_mv | AT yilin realtimecontrollingdynamicssensinginairtrafficsystem AT xianlongtan realtimecontrollingdynamicssensinginairtrafficsystem AT boyang realtimecontrollingdynamicssensinginairtrafficsystem AT kaiyang realtimecontrollingdynamicssensinginairtrafficsystem AT jianweizhang realtimecontrollingdynamicssensinginairtrafficsystem AT jingyu realtimecontrollingdynamicssensinginairtrafficsystem |