Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition
To realize an early warning of unbalanced workload in the aircraft cockpit, it is required to monitor the pilot’s real-time workload condition. For the purpose of building the mapping relationship from physiological and flight data to workload, a multi-source data fusion model is proposed...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/16/3629 |
| _version_ | 1828115157832695808 |
|---|---|
| author | Xia Zhang Youchao Sun Zhifan Qiu Junping Bao Yanjun Zhang |
| author_facet | Xia Zhang Youchao Sun Zhifan Qiu Junping Bao Yanjun Zhang |
| author_sort | Xia Zhang |
| collection | DOAJ |
| description | To realize an early warning of unbalanced workload in the aircraft cockpit, it is required to monitor the pilot’s real-time workload condition. For the purpose of building the mapping relationship from physiological and flight data to workload, a multi-source data fusion model is proposed based on a fuzzy neural network, mainly structured using a principal components extraction layer, fuzzification layer, fuzzy rules matching layer, and normalization layer. Aiming at the high coupling characteristic variables contributing to workload, principal component analysis reconstructs the feature data by reducing its dimension. Considering the uncertainty for a single variable to reflect overall workload, a fuzzy membership function and fuzzy control rules are defined to abstract the inference process. An error feedforward algorithm based on gradient descent is utilized for parameter learning. Convergence speed and accuracy can be adjusted by controlling the gradient descent rate and error tolerance threshold. Combined with takeoff and initial climbing tasks of a Boeing 737−800 aircraft, crucial performance indicators—including pitch angle, heading, and airspeed—as well as physiological indicators—including electrocardiogram (ECG), respiration, and eye movements—were featured. The mapping relationship between multi-source data and the comprehensive workload level synthesized using the NASA task load index was established. Experimental results revealed that the predicted workload corresponding to different flight phases and difficulty levels showed clear distinctions, thereby proving the validity of data fusion. |
| first_indexed | 2024-04-11T12:36:30Z |
| format | Article |
| id | doaj.art-33b679c50a31438c963deaeeaad1ca45 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T12:36:30Z |
| publishDate | 2019-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-33b679c50a31438c963deaeeaad1ca452022-12-22T04:23:37ZengMDPI AGSensors1424-82202019-08-011916362910.3390/s19163629s19163629Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload ConditionXia Zhang0Youchao Sun1Zhifan Qiu2Junping Bao3Yanjun Zhang4College of Civil Aviation, Nanjing University of Aeronautics & Astronautics, Nanjing 211106, ChinaCollege of Civil Aviation, Nanjing University of Aeronautics & Astronautics, Nanjing 211106, ChinaShanghai Aircraft Design & Research Institute, Commercial Aircraft Corporation of China, Ltd., Shanghai 201210, ChinaShanghai Aircraft Design & Research Institute, Commercial Aircraft Corporation of China, Ltd., Shanghai 201210, ChinaSchool of Mechanical Engineering, Yangzhou University, Yangzhou 225127, ChinaTo realize an early warning of unbalanced workload in the aircraft cockpit, it is required to monitor the pilot’s real-time workload condition. For the purpose of building the mapping relationship from physiological and flight data to workload, a multi-source data fusion model is proposed based on a fuzzy neural network, mainly structured using a principal components extraction layer, fuzzification layer, fuzzy rules matching layer, and normalization layer. Aiming at the high coupling characteristic variables contributing to workload, principal component analysis reconstructs the feature data by reducing its dimension. Considering the uncertainty for a single variable to reflect overall workload, a fuzzy membership function and fuzzy control rules are defined to abstract the inference process. An error feedforward algorithm based on gradient descent is utilized for parameter learning. Convergence speed and accuracy can be adjusted by controlling the gradient descent rate and error tolerance threshold. Combined with takeoff and initial climbing tasks of a Boeing 737−800 aircraft, crucial performance indicators—including pitch angle, heading, and airspeed—as well as physiological indicators—including electrocardiogram (ECG), respiration, and eye movements—were featured. The mapping relationship between multi-source data and the comprehensive workload level synthesized using the NASA task load index was established. Experimental results revealed that the predicted workload corresponding to different flight phases and difficulty levels showed clear distinctions, thereby proving the validity of data fusion.https://www.mdpi.com/1424-8220/19/16/3629aircraft pilotworkloadmulti-source data fusionfuzzy neural networkprincipal component analysisparameter learning |
| spellingShingle | Xia Zhang Youchao Sun Zhifan Qiu Junping Bao Yanjun Zhang Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition Sensors aircraft pilot workload multi-source data fusion fuzzy neural network principal component analysis parameter learning |
| title | Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition |
| title_full | Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition |
| title_fullStr | Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition |
| title_full_unstemmed | Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition |
| title_short | Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot’s Workload Condition |
| title_sort | adaptive neuro fuzzy fusion of multi sensor data for monitoring a pilot s workload condition |
| topic | aircraft pilot workload multi-source data fusion fuzzy neural network principal component analysis parameter learning |
| url | https://www.mdpi.com/1424-8220/19/16/3629 |
| work_keys_str_mv | AT xiazhang adaptiveneurofuzzyfusionofmultisensordataformonitoringapilotsworkloadcondition AT youchaosun adaptiveneurofuzzyfusionofmultisensordataformonitoringapilotsworkloadcondition AT zhifanqiu adaptiveneurofuzzyfusionofmultisensordataformonitoringapilotsworkloadcondition AT junpingbao adaptiveneurofuzzyfusionofmultisensordataformonitoringapilotsworkloadcondition AT yanjunzhang adaptiveneurofuzzyfusionofmultisensordataformonitoringapilotsworkloadcondition |