<i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation
Driver monitoring systems play an important role in lower to mid-level autonomous vehicles. Our work focuses on the detection of cognitive load as a component of driver-state estimation to improve traffic safety. By inducing single and dual-task workloads of increasing intensity on 51 subjects, whil...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/1/340 |
| _version_ | 1797439594547904512 |
|---|---|
| author | Maximilian P. Oppelt Andreas Foltyn Jessica Deuschel Nadine R. Lang Nina Holzer Bjoern M. Eskofier Seung Hee Yang |
| author_facet | Maximilian P. Oppelt Andreas Foltyn Jessica Deuschel Nadine R. Lang Nina Holzer Bjoern M. Eskofier Seung Hee Yang |
| author_sort | Maximilian P. Oppelt |
| collection | DOAJ |
| description | Driver monitoring systems play an important role in lower to mid-level autonomous vehicles. Our work focuses on the detection of cognitive load as a component of driver-state estimation to improve traffic safety. By inducing single and dual-task workloads of increasing intensity on 51 subjects, while continuously measuring signals from multiple modalities, based on <i>physiological</i> measurements such as ECG, EDA, EMG, PPG, respiration rate, skin temperature and eye tracker data, as well as <i>behavioral</i> measurements such as action units extracted from facial videos, <i>performance</i> metrics like reaction time and <i>subjective</i> feedback using questionnaires, we create <i>ADABase</i> (<b>A</b>utonomous <b>D</b>riving Cognitive Load <b>A</b>ssessment Data<b>base</b>) As a reference method to induce cognitive load onto subjects, we use the well-established <i>n</i>-back test, in addition to our novel simulator-based <i>k</i>-drive test, motivated by real-world semi-autonomously vehicles. We extract expert features of all measurements and find significant changes in multiple modalities. Ultimately we train and evaluate machine learning algorithms using single and multimodal inputs to distinguish cognitive load levels. We carefully evaluate model behavior and study feature importance. In summary, we introduce a novel cognitive load test, create a cognitive load database, validate changes using statistical tests, introduce novel classification and regression tasks for machine learning and train and evaluate machine learning models. |
| first_indexed | 2024-03-09T11:56:20Z |
| format | Article |
| id | doaj.art-2d9d37abb4f143d29a67c67e47b341e2 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T11:56:20Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-2d9d37abb4f143d29a67c67e47b341e22023-11-30T23:08:55ZengMDPI AGSensors1424-82202022-12-0123134010.3390/s23010340<i>ADABase</i>: A Multimodal Dataset for Cognitive Load EstimationMaximilian P. Oppelt0Andreas Foltyn1Jessica Deuschel2Nadine R. Lang3Nina Holzer4Bjoern M. Eskofier5Seung Hee Yang6Department Digital Health Systems, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, 91058 Erlangen, GermanyDepartment Sensory Perception and Analytics, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, 91058 Erlangen, GermanyDepartment Sensory Perception and Analytics, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, 91058 Erlangen, GermanyDepartment Digital Health Systems, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, 91058 Erlangen, GermanyDepartment Sensory Perception and Analytics, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, 91058 Erlangen, GermanyMachine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen Nuremberg, 91052 Erlangen, GermanyArtificial Intelligence in Biomedical Speech Processing Lab, Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen Nuremberg, 91052 Erlangen, GermanyDriver monitoring systems play an important role in lower to mid-level autonomous vehicles. Our work focuses on the detection of cognitive load as a component of driver-state estimation to improve traffic safety. By inducing single and dual-task workloads of increasing intensity on 51 subjects, while continuously measuring signals from multiple modalities, based on <i>physiological</i> measurements such as ECG, EDA, EMG, PPG, respiration rate, skin temperature and eye tracker data, as well as <i>behavioral</i> measurements such as action units extracted from facial videos, <i>performance</i> metrics like reaction time and <i>subjective</i> feedback using questionnaires, we create <i>ADABase</i> (<b>A</b>utonomous <b>D</b>riving Cognitive Load <b>A</b>ssessment Data<b>base</b>) As a reference method to induce cognitive load onto subjects, we use the well-established <i>n</i>-back test, in addition to our novel simulator-based <i>k</i>-drive test, motivated by real-world semi-autonomously vehicles. We extract expert features of all measurements and find significant changes in multiple modalities. Ultimately we train and evaluate machine learning algorithms using single and multimodal inputs to distinguish cognitive load levels. We carefully evaluate model behavior and study feature importance. In summary, we introduce a novel cognitive load test, create a cognitive load database, validate changes using statistical tests, introduce novel classification and regression tasks for machine learning and train and evaluate machine learning models.https://www.mdpi.com/1424-8220/23/1/340cognitive loadaffective computingautonomous drivingmachine learningmultimodal dataset |
| spellingShingle | Maximilian P. Oppelt Andreas Foltyn Jessica Deuschel Nadine R. Lang Nina Holzer Bjoern M. Eskofier Seung Hee Yang <i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation Sensors cognitive load affective computing autonomous driving machine learning multimodal dataset |
| title | <i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation |
| title_full | <i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation |
| title_fullStr | <i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation |
| title_full_unstemmed | <i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation |
| title_short | <i>ADABase</i>: A Multimodal Dataset for Cognitive Load Estimation |
| title_sort | i adabase i a multimodal dataset for cognitive load estimation |
| topic | cognitive load affective computing autonomous driving machine learning multimodal dataset |
| url | https://www.mdpi.com/1424-8220/23/1/340 |
| work_keys_str_mv | AT maximilianpoppelt iadabaseiamultimodaldatasetforcognitiveloadestimation AT andreasfoltyn iadabaseiamultimodaldatasetforcognitiveloadestimation AT jessicadeuschel iadabaseiamultimodaldatasetforcognitiveloadestimation AT nadinerlang iadabaseiamultimodaldatasetforcognitiveloadestimation AT ninaholzer iadabaseiamultimodaldatasetforcognitiveloadestimation AT bjoernmeskofier iadabaseiamultimodaldatasetforcognitiveloadestimation AT seungheeyang iadabaseiamultimodaldatasetforcognitiveloadestimation |