Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks
Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cr...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2018-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1424-8220/18/5/1339 |
| _version_ | 1818040256566394880 |
|---|---|
| author | Ryan Hefron Brett Borghetti Christine Schubert Kabban James Christensen Justin Estepp |
| author_facet | Ryan Hefron Brett Borghetti Christine Schubert Kabban James Christensen Justin Estepp |
| author_sort | Ryan Hefron |
| collection | DOAJ |
| description | Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals’ EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance. |
| first_indexed | 2024-12-10T08:11:38Z |
| format | Article |
| id | doaj.art-da014debae9f495cb15d8e74c524964a |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-12-10T08:11:38Z |
| publishDate | 2018-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-da014debae9f495cb15d8e74c524964a2022-12-22T01:56:33ZengMDPI AGSensors1424-82202018-04-01185133910.3390/s18051339s18051339Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural NetworksRyan Hefron0Brett Borghetti1Christine Schubert Kabban2James Christensen3Justin Estepp4Department of Electrical & Computer Engineering, Air Force Institute of Technology, WPAFB, Dayton, OH 45433, USADepartment of Electrical & Computer Engineering, Air Force Institute of Technology, WPAFB, Dayton, OH 45433, USADepartment of Mathematics & Statistics, Air Force Institute of Technology, WPAFB, Dayton, OH 45433, USAAir Force Research Laboratory, WPAFB, Dayton, OH 45433, USAAir Force Research Laboratory, WPAFB, Dayton, OH 45433, USAApplying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals’ EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance.http://www.mdpi.com/1424-8220/18/5/1339convolutionalrecurrentneural networkcognitive workloadMATBEEGcross-participantmental workloadtemporal specificityensemble |
| spellingShingle | Ryan Hefron Brett Borghetti Christine Schubert Kabban James Christensen Justin Estepp Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks Sensors convolutional recurrent neural network cognitive workload MATB EEG cross-participant mental workload temporal specificity ensemble |
| title | Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks |
| title_full | Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks |
| title_fullStr | Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks |
| title_full_unstemmed | Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks |
| title_short | Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks |
| title_sort | cross participant eeg based assessment of cognitive workload using multi path convolutional recurrent neural networks |
| topic | convolutional recurrent neural network cognitive workload MATB EEG cross-participant mental workload temporal specificity ensemble |
| url | http://www.mdpi.com/1424-8220/18/5/1339 |
| work_keys_str_mv | AT ryanhefron crossparticipanteegbasedassessmentofcognitiveworkloadusingmultipathconvolutionalrecurrentneuralnetworks AT brettborghetti crossparticipanteegbasedassessmentofcognitiveworkloadusingmultipathconvolutionalrecurrentneuralnetworks AT christineschubertkabban crossparticipanteegbasedassessmentofcognitiveworkloadusingmultipathconvolutionalrecurrentneuralnetworks AT jameschristensen crossparticipanteegbasedassessmentofcognitiveworkloadusingmultipathconvolutionalrecurrentneuralnetworks AT justinestepp crossparticipanteegbasedassessmentofcognitiveworkloadusingmultipathconvolutionalrecurrentneuralnetworks |