Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers
In order to determine the fatigue state of air traffic controllers from air talk, an algorithm is proposed for discriminating the fatigue state of controllers based on applying multi-speech feature fusion to voice data using a Fuzzy Support Vector Machine (FSVM). To supplement the basis for discrimi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Aerospace |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2226-4310/11/1/15 |
| _version_ | 1797344906196287488 |
|---|---|
| author | Lin Xu Shanxiu Ma Zhiyuan Shen Shiyu Huang Ying Nan |
| author_facet | Lin Xu Shanxiu Ma Zhiyuan Shen Shiyu Huang Ying Nan |
| author_sort | Lin Xu |
| collection | DOAJ |
| description | In order to determine the fatigue state of air traffic controllers from air talk, an algorithm is proposed for discriminating the fatigue state of controllers based on applying multi-speech feature fusion to voice data using a Fuzzy Support Vector Machine (FSVM). To supplement the basis for discrimination, we also extracted eye-fatigue-state discrimination features based on Percentage of Eyelid Closure Duration (PERCLOS) eye data. To merge the two classes of discrimination results, a new controller fatigue-state evaluation index based on the entropy weight method is proposed, based on a decision-level fusion of fatigue discrimination results for speech and the eyes. The experimental results show that the fatigue-state recognition accuracy rate was 86.0% for the fatigue state evaluation index, which was 3.5% and 2.2%higher than those for speech and eye assessments, respectively. The comprehensive fatigue evaluation index provides important reference values for controller scheduling and mental-state evaluations. |
| first_indexed | 2024-03-08T11:09:40Z |
| format | Article |
| id | doaj.art-b8c1c783756e45af81f4142f59542e5e |
| institution | Directory Open Access Journal |
| issn | 2226-4310 |
| language | English |
| last_indexed | 2024-03-08T11:09:40Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Aerospace |
| spelling | doaj.art-b8c1c783756e45af81f4142f59542e5e2024-01-26T14:11:34ZengMDPI AGAerospace2226-43102023-12-011111510.3390/aerospace11010015Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic ControllersLin Xu0Shanxiu Ma1Zhiyuan Shen2Shiyu Huang3Ying Nan4College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaIn order to determine the fatigue state of air traffic controllers from air talk, an algorithm is proposed for discriminating the fatigue state of controllers based on applying multi-speech feature fusion to voice data using a Fuzzy Support Vector Machine (FSVM). To supplement the basis for discrimination, we also extracted eye-fatigue-state discrimination features based on Percentage of Eyelid Closure Duration (PERCLOS) eye data. To merge the two classes of discrimination results, a new controller fatigue-state evaluation index based on the entropy weight method is proposed, based on a decision-level fusion of fatigue discrimination results for speech and the eyes. The experimental results show that the fatigue-state recognition accuracy rate was 86.0% for the fatigue state evaluation index, which was 3.5% and 2.2%higher than those for speech and eye assessments, respectively. The comprehensive fatigue evaluation index provides important reference values for controller scheduling and mental-state evaluations.https://www.mdpi.com/2226-4310/11/1/15fatigue recognitionair traffic controllerfeature fusionmulti-mode |
| spellingShingle | Lin Xu Shanxiu Ma Zhiyuan Shen Shiyu Huang Ying Nan Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers Aerospace fatigue recognition air traffic controller feature fusion multi-mode |
| title | Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers |
| title_full | Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers |
| title_fullStr | Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers |
| title_full_unstemmed | Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers |
| title_short | Analyzing Multi-Mode Fatigue Information from Speech and Gaze Data from Air Traffic Controllers |
| title_sort | analyzing multi mode fatigue information from speech and gaze data from air traffic controllers |
| topic | fatigue recognition air traffic controller feature fusion multi-mode |
| url | https://www.mdpi.com/2226-4310/11/1/15 |
| work_keys_str_mv | AT linxu analyzingmultimodefatigueinformationfromspeechandgazedatafromairtrafficcontrollers AT shanxiuma analyzingmultimodefatigueinformationfromspeechandgazedatafromairtrafficcontrollers AT zhiyuanshen analyzingmultimodefatigueinformationfromspeechandgazedatafromairtrafficcontrollers AT shiyuhuang analyzingmultimodefatigueinformationfromspeechandgazedatafromairtrafficcontrollers AT yingnan analyzingmultimodefatigueinformationfromspeechandgazedatafromairtrafficcontrollers |