Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study
Trackers for activity and physical fitness have become ubiquitous. Although recent work has demonstrated significant relationships between mental effort and physiological data such as skin temperature, heart rate, and electrodermal activity, we have yet to demonstrate their efficacy for the forecast...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Signals |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2624-6120/2/4/51 |
| _version_ | 1827669603266854912 |
|---|---|
| author | Ankita Agarwal Josephine Graft Noah Schroeder William Romine |
| author_facet | Ankita Agarwal Josephine Graft Noah Schroeder William Romine |
| author_sort | Ankita Agarwal |
| collection | DOAJ |
| description | Trackers for activity and physical fitness have become ubiquitous. Although recent work has demonstrated significant relationships between mental effort and physiological data such as skin temperature, heart rate, and electrodermal activity, we have yet to demonstrate their efficacy for the forecasting of mental effort such that a useful mental effort tracker can be developed. Given prior difficulty in extracting relationships between mental effort and physiological responses that are repeatable across individuals, we make the case that fusing self-report measures with physiological data within an internet or smartphone application may provide an effective method for training a useful mental effort tracking system. In this case study, we utilized over 90 h of data from a single participant over the course of a college semester. By fusing the participant’s self-reported mental effort in different activities over the course of the semester with concurrent physiological data collected with the Empatica E4 wearable sensor, we explored questions around how much data were needed to train such a device, and which types of machine-learning algorithms worked best. We concluded that although baseline models such as logistic regression and Markov models provided useful explanatory information on how the student’s physiology changed with mental effort, deep-learning algorithms were able to generate accurate predictions using the first 28 h of data for training. A system that combines long short-term memory and convolutional neural networks is recommended in order to generate smooth predictions while also being able to capture transitions in mental effort when they occur in the individual using the device. |
| first_indexed | 2024-03-10T03:05:33Z |
| format | Article |
| id | doaj.art-30ae507d7a264b78b3857fc014d93738 |
| institution | Directory Open Access Journal |
| issn | 2624-6120 |
| language | English |
| last_indexed | 2024-03-10T03:05:33Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Signals |
| spelling | doaj.art-30ae507d7a264b78b3857fc014d937382023-11-23T10:33:10ZengMDPI AGSignals2624-61202021-12-012488690110.3390/signals2040051Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case StudyAnkita Agarwal0Josephine Graft1Noah Schroeder2William Romine3Department of Computer Science and Engineering, Wright State University, Dayton, OH 45435, USADepartment of Biological Sciences, Wright State University, Dayton, OH 45435, USADepartment of Leadership Studies in Education and Organizations, Wright State University, Dayton, OH 45435, USADepartment of Biological Sciences, Wright State University, Dayton, OH 45435, USATrackers for activity and physical fitness have become ubiquitous. Although recent work has demonstrated significant relationships between mental effort and physiological data such as skin temperature, heart rate, and electrodermal activity, we have yet to demonstrate their efficacy for the forecasting of mental effort such that a useful mental effort tracker can be developed. Given prior difficulty in extracting relationships between mental effort and physiological responses that are repeatable across individuals, we make the case that fusing self-report measures with physiological data within an internet or smartphone application may provide an effective method for training a useful mental effort tracking system. In this case study, we utilized over 90 h of data from a single participant over the course of a college semester. By fusing the participant’s self-reported mental effort in different activities over the course of the semester with concurrent physiological data collected with the Empatica E4 wearable sensor, we explored questions around how much data were needed to train such a device, and which types of machine-learning algorithms worked best. We concluded that although baseline models such as logistic regression and Markov models provided useful explanatory information on how the student’s physiology changed with mental effort, deep-learning algorithms were able to generate accurate predictions using the first 28 h of data for training. A system that combines long short-term memory and convolutional neural networks is recommended in order to generate smooth predictions while also being able to capture transitions in mental effort when they occur in the individual using the device.https://www.mdpi.com/2624-6120/2/4/51cognitive loadmental effortdeep learningwearable sensorlearning analytics |
| spellingShingle | Ankita Agarwal Josephine Graft Noah Schroeder William Romine Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study Signals cognitive load mental effort deep learning wearable sensor learning analytics |
| title | Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study |
| title_full | Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study |
| title_fullStr | Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study |
| title_full_unstemmed | Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study |
| title_short | Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study |
| title_sort | sensor based prediction of mental effort during learning from physiological data a longitudinal case study |
| topic | cognitive load mental effort deep learning wearable sensor learning analytics |
| url | https://www.mdpi.com/2624-6120/2/4/51 |
| work_keys_str_mv | AT ankitaagarwal sensorbasedpredictionofmentaleffortduringlearningfromphysiologicaldataalongitudinalcasestudy AT josephinegraft sensorbasedpredictionofmentaleffortduringlearningfromphysiologicaldataalongitudinalcasestudy AT noahschroeder sensorbasedpredictionofmentaleffortduringlearningfromphysiologicaldataalongitudinalcasestudy AT williamromine sensorbasedpredictionofmentaleffortduringlearningfromphysiologicaldataalongitudinalcasestudy |