Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network
Non-invasive brain-computer interface (BCI) has been developed for recognizing and classifying human mental states with high performances. Specifically, classifying pilots' mental states accurately is a critical issue because their cognitive states, which are induced by mental fatigue, workload...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9133061/ |
| _version_ | 1818330308285562880 |
|---|---|
| author | Dae-Hyeok Lee Ji-Hoon Jeong Kiduk Kim Baek-Woon Yu Seong-Whan Lee |
| author_facet | Dae-Hyeok Lee Ji-Hoon Jeong Kiduk Kim Baek-Woon Yu Seong-Whan Lee |
| author_sort | Dae-Hyeok Lee |
| collection | DOAJ |
| description | Non-invasive brain-computer interface (BCI) has been developed for recognizing and classifying human mental states with high performances. Specifically, classifying pilots' mental states accurately is a critical issue because their cognitive states, which are induced by mental fatigue, workload, and distraction, may be fundamental in catastrophic accidents. In this study, we present an electroencephalogram (EEG) classification of four mental states (fatigue, workload, distraction, and the normal state) from EEG signals in both offline and pseudo-online analyses. To the best of our knowledge, this study is the first attempt to classify pilots' mental states using only EEG signals during continuous decoding. We recorded EEG signals from seven pilots under various simulated flight conditions. We proposed a multiple feature block-based convolutional neural network (MFB-CNN) with temporal-spatio EEG filters to recognize the pilot's current mental states. We validated the proposed method for two analyses across all subjects. In the offline analysis, we confirmed the classification accuracy of 0.75 (±0.04). Also, in the pseudo-online analysis, we obtained the detection accuracy of 0.72 (±0.20), 0.72 (±0.27), and 0.61 (±0.18) for fatigue, workload, and distraction, respectively. Hence, we demonstrate the feasibility of classifying various types of mental states for implementation in real-world environments. |
| first_indexed | 2024-12-13T13:01:53Z |
| format | Article |
| id | doaj.art-62e0717b791b47858dfed07b55f2e291 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T13:01:53Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-62e0717b791b47858dfed07b55f2e2912022-12-21T23:44:59ZengIEEEIEEE Access2169-35362020-01-01812192912194110.1109/ACCESS.2020.30069079133061Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural NetworkDae-Hyeok Lee0https://orcid.org/0000-0002-2238-8910Ji-Hoon Jeong1https://orcid.org/0000-0001-6940-2700Kiduk Kim2Baek-Woon Yu3Seong-Whan Lee4https://orcid.org/0000-0002-6249-4996Department of Brain and Cognitive Engineering, Korea University, Seoul, South KoreaDepartment of Brain and Cognitive Engineering, Korea University, Seoul, South KoreaDepartment of Brain and Cognitive Engineering, Korea University, Seoul, South KoreaDepartment of Brain and Cognitive Engineering, Korea University, Seoul, South KoreaDepartment of Artificial Intelligence, Korea University, Seoul, South KoreaNon-invasive brain-computer interface (BCI) has been developed for recognizing and classifying human mental states with high performances. Specifically, classifying pilots' mental states accurately is a critical issue because their cognitive states, which are induced by mental fatigue, workload, and distraction, may be fundamental in catastrophic accidents. In this study, we present an electroencephalogram (EEG) classification of four mental states (fatigue, workload, distraction, and the normal state) from EEG signals in both offline and pseudo-online analyses. To the best of our knowledge, this study is the first attempt to classify pilots' mental states using only EEG signals during continuous decoding. We recorded EEG signals from seven pilots under various simulated flight conditions. We proposed a multiple feature block-based convolutional neural network (MFB-CNN) with temporal-spatio EEG filters to recognize the pilot's current mental states. We validated the proposed method for two analyses across all subjects. In the offline analysis, we confirmed the classification accuracy of 0.75 (±0.04). Also, in the pseudo-online analysis, we obtained the detection accuracy of 0.72 (±0.20), 0.72 (±0.27), and 0.61 (±0.18) for fatigue, workload, and distraction, respectively. Hence, we demonstrate the feasibility of classifying various types of mental states for implementation in real-world environments.https://ieeexplore.ieee.org/document/9133061/Brain-computer interface (BCI)electroencephalogram (EEG)mental statesdeep convolutional neural network |
| spellingShingle | Dae-Hyeok Lee Ji-Hoon Jeong Kiduk Kim Baek-Woon Yu Seong-Whan Lee Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network IEEE Access Brain-computer interface (BCI) electroencephalogram (EEG) mental states deep convolutional neural network |
| title | Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network |
| title_full | Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network |
| title_fullStr | Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network |
| title_full_unstemmed | Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network |
| title_short | Continuous EEG Decoding of Pilots’ Mental States Using Multiple Feature Block-Based Convolutional Neural Network |
| title_sort | continuous eeg decoding of pilots x2019 mental states using multiple feature block based convolutional neural network |
| topic | Brain-computer interface (BCI) electroencephalogram (EEG) mental states deep convolutional neural network |
| url | https://ieeexplore.ieee.org/document/9133061/ |
| work_keys_str_mv | AT daehyeoklee continuouseegdecodingofpilotsx2019mentalstatesusingmultiplefeatureblockbasedconvolutionalneuralnetwork AT jihoonjeong continuouseegdecodingofpilotsx2019mentalstatesusingmultiplefeatureblockbasedconvolutionalneuralnetwork AT kidukkim continuouseegdecodingofpilotsx2019mentalstatesusingmultiplefeatureblockbasedconvolutionalneuralnetwork AT baekwoonyu continuouseegdecodingofpilotsx2019mentalstatesusingmultiplefeatureblockbasedconvolutionalneuralnetwork AT seongwhanlee continuouseegdecodingofpilotsx2019mentalstatesusingmultiplefeatureblockbasedconvolutionalneuralnetwork |