Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling
This study presents the integration of a passive brain-computer interface (pBCI) and cognitive modeling as a method to trace pilots’ perception and processing of auditory alerts and messages during operations. Missing alerts on the flight deck can result in out-of-the-loop problems that can lead to...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2020-08-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnins.2020.00795/full |
| _version_ | 1818837008808673280 |
|---|---|
| author | Oliver W. Klaproth Oliver W. Klaproth Christoph Vernaleken Laurens R. Krol Marc Halbruegge Thorsten O. Zander Thorsten O. Zander Nele Russwinkel |
| author_facet | Oliver W. Klaproth Oliver W. Klaproth Christoph Vernaleken Laurens R. Krol Marc Halbruegge Thorsten O. Zander Thorsten O. Zander Nele Russwinkel |
| author_sort | Oliver W. Klaproth |
| collection | DOAJ |
| description | This study presents the integration of a passive brain-computer interface (pBCI) and cognitive modeling as a method to trace pilots’ perception and processing of auditory alerts and messages during operations. Missing alerts on the flight deck can result in out-of-the-loop problems that can lead to accidents. By tracing pilots’ perception and responses to alerts, cognitive assistance can be provided based on individual needs to ensure they maintain adequate situation awareness. Data from 24 participating aircrew in a simulated flight study that included multiple alerts and air traffic control messages in single pilot setup are presented. A classifier was trained to identify pilots’ neurophysiological reactions to alerts and messages from participants’ electroencephalogram (EEG). A neuroadaptive ACT-R model using EEG data was compared to a conventional normative model regarding accuracy in representing individual pilots. Results show that passive BCI can distinguish between alerts that are processed by the pilot as task-relevant or irrelevant in the cockpit based on the recorded EEG. The neuroadaptive model’s integration of this data resulted in significantly higher performance of 87% overall accuracy in representing individual pilots’ responses to alerts and messages compared to 72% accuracy of a normative model that did not consider EEG data. We conclude that neuroadaptive technology allows for implicit measurement and tracing of pilots’ perception and processing of alerts on the flight deck. Careful handling of uncertainties inherent to passive BCI and cognitive modeling shows how the representation of pilot cognitive states can be improved iteratively for providing assistance. |
| first_indexed | 2024-12-19T03:15:41Z |
| format | Article |
| id | doaj.art-5479430df6b1439f8a6afbb8dcca2a99 |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-19T03:15:41Z |
| publishDate | 2020-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-5479430df6b1439f8a6afbb8dcca2a992022-12-21T20:37:54ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2020-08-011410.3389/fnins.2020.00795560321Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive ModelingOliver W. Klaproth0Oliver W. Klaproth1Christoph Vernaleken2Laurens R. Krol3Marc Halbruegge4Thorsten O. Zander5Thorsten O. Zander6Nele Russwinkel7Airbus Central R&T, Hamburg, GermanyChair of Cognitive Modelling in Dynamic Systems, Department of Psychology and Ergonomics, Technische Universität Berlin, Berlin, GermanyAirbus Defence and Space, Manching, GermanyZander Laboratories B.V., Amsterdam, NetherlandsChair of Cognitive Modelling in Dynamic Systems, Department of Psychology and Ergonomics, Technische Universität Berlin, Berlin, GermanyZander Laboratories B.V., Amsterdam, NetherlandsChair of Neuroadaptive Human-Computer Interaction, Brandenburg University of Technology, Cottbus-Senftenberg, GermanyChair of Cognitive Modelling in Dynamic Systems, Department of Psychology and Ergonomics, Technische Universität Berlin, Berlin, GermanyThis study presents the integration of a passive brain-computer interface (pBCI) and cognitive modeling as a method to trace pilots’ perception and processing of auditory alerts and messages during operations. Missing alerts on the flight deck can result in out-of-the-loop problems that can lead to accidents. By tracing pilots’ perception and responses to alerts, cognitive assistance can be provided based on individual needs to ensure they maintain adequate situation awareness. Data from 24 participating aircrew in a simulated flight study that included multiple alerts and air traffic control messages in single pilot setup are presented. A classifier was trained to identify pilots’ neurophysiological reactions to alerts and messages from participants’ electroencephalogram (EEG). A neuroadaptive ACT-R model using EEG data was compared to a conventional normative model regarding accuracy in representing individual pilots. Results show that passive BCI can distinguish between alerts that are processed by the pilot as task-relevant or irrelevant in the cockpit based on the recorded EEG. The neuroadaptive model’s integration of this data resulted in significantly higher performance of 87% overall accuracy in representing individual pilots’ responses to alerts and messages compared to 72% accuracy of a normative model that did not consider EEG data. We conclude that neuroadaptive technology allows for implicit measurement and tracing of pilots’ perception and processing of alerts on the flight deck. Careful handling of uncertainties inherent to passive BCI and cognitive modeling shows how the representation of pilot cognitive states can be improved iteratively for providing assistance.https://www.frontiersin.org/article/10.3389/fnins.2020.00795/fullsituation awarenessaviationbrain-computer-interfacesACT-Rhuman-automation interaction |
| spellingShingle | Oliver W. Klaproth Oliver W. Klaproth Christoph Vernaleken Laurens R. Krol Marc Halbruegge Thorsten O. Zander Thorsten O. Zander Nele Russwinkel Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling Frontiers in Neuroscience situation awareness aviation brain-computer-interfaces ACT-R human-automation interaction |
| title | Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling |
| title_full | Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling |
| title_fullStr | Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling |
| title_full_unstemmed | Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling |
| title_short | Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling |
| title_sort | tracing pilots situation assessment by neuroadaptive cognitive modeling |
| topic | situation awareness aviation brain-computer-interfaces ACT-R human-automation interaction |
| url | https://www.frontiersin.org/article/10.3389/fnins.2020.00795/full |
| work_keys_str_mv | AT oliverwklaproth tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT oliverwklaproth tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT christophvernaleken tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT laurensrkrol tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT marchalbruegge tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT thorstenozander tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT thorstenozander tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling AT nelerusswinkel tracingpilotssituationassessmentbyneuroadaptivecognitivemodeling |