Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features
Emotion recognition research has been conducted using various physiological signals. In this paper, we propose an efficient photoplethysmogram-based method that fuses the deep features extracted by two deep convolutional neural networks and the statistical features selected by Pearson’s correlation...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-05-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/10/10/3501 |
| _version_ | 1797567583785844736 |
|---|---|
| author | MinSeop Lee Yun Kyu Lee Myo-Taeg Lim Tae-Koo Kang |
| author_facet | MinSeop Lee Yun Kyu Lee Myo-Taeg Lim Tae-Koo Kang |
| author_sort | MinSeop Lee |
| collection | DOAJ |
| description | Emotion recognition research has been conducted using various physiological signals. In this paper, we propose an efficient photoplethysmogram-based method that fuses the deep features extracted by two deep convolutional neural networks and the statistical features selected by Pearson’s correlation technique. A photoplethysmogram (PPG) signal can be easily obtained through many devices, and the procedure for recording this signal is simpler than that for other physiological signals. The normal-to-normal (NN) interval values of heart rate variability (HRV) were utilized to extract the time domain features, and the normalized PPG signal was used to acquire the frequency domain features. Then, we selected features that correlated highly with an emotion through Pearson’s correlation. These statistical features were fused with deep-learning features extracted from a convolutional neural network (CNN). The PPG signal and the NN interval were used as the inputs of the CNN to extract the features, and the total concatenated features were utilized to classify the valence and the arousal, which are the basic parameters of emotion. The Database for Emotion Analysis using Physiological signals (DEAP) was chosen for the experiment, and the results demonstrated that the proposed method achieved a noticeable performance with a short recognition interval. |
| first_indexed | 2024-03-10T19:44:58Z |
| format | Article |
| id | doaj.art-9840a381dd964a27917a175603ab633e |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T19:44:58Z |
| publishDate | 2020-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-9840a381dd964a27917a175603ab633e2023-11-20T00:55:00ZengMDPI AGApplied Sciences2076-34172020-05-011010350110.3390/app10103501Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram FeaturesMinSeop Lee0Yun Kyu Lee1Myo-Taeg Lim2Tae-Koo Kang3School of Electrical Engineering, Korea University, Seoul 02841, KoreaSchool of Electrical Engineering, Korea University, Seoul 02841, KoreaSchool of Electrical Engineering, Korea University, Seoul 02841, KoreaDepartment of Human Intelligence and Robot Engineering, Sangmyung University, Cheonan 31066, KoreaEmotion recognition research has been conducted using various physiological signals. In this paper, we propose an efficient photoplethysmogram-based method that fuses the deep features extracted by two deep convolutional neural networks and the statistical features selected by Pearson’s correlation technique. A photoplethysmogram (PPG) signal can be easily obtained through many devices, and the procedure for recording this signal is simpler than that for other physiological signals. The normal-to-normal (NN) interval values of heart rate variability (HRV) were utilized to extract the time domain features, and the normalized PPG signal was used to acquire the frequency domain features. Then, we selected features that correlated highly with an emotion through Pearson’s correlation. These statistical features were fused with deep-learning features extracted from a convolutional neural network (CNN). The PPG signal and the NN interval were used as the inputs of the CNN to extract the features, and the total concatenated features were utilized to classify the valence and the arousal, which are the basic parameters of emotion. The Database for Emotion Analysis using Physiological signals (DEAP) was chosen for the experiment, and the results demonstrated that the proposed method achieved a noticeable performance with a short recognition interval.https://www.mdpi.com/2076-3417/10/10/3501PPGemotion recognitionstatistical featurefeature fusionconvolutional neural network |
| spellingShingle | MinSeop Lee Yun Kyu Lee Myo-Taeg Lim Tae-Koo Kang Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features Applied Sciences PPG emotion recognition statistical feature feature fusion convolutional neural network |
| title | Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features |
| title_full | Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features |
| title_fullStr | Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features |
| title_full_unstemmed | Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features |
| title_short | Emotion Recognition Using Convolutional Neural Network with Selected Statistical Photoplethysmogram Features |
| title_sort | emotion recognition using convolutional neural network with selected statistical photoplethysmogram features |
| topic | PPG emotion recognition statistical feature feature fusion convolutional neural network |
| url | https://www.mdpi.com/2076-3417/10/10/3501 |
| work_keys_str_mv | AT minseoplee emotionrecognitionusingconvolutionalneuralnetworkwithselectedstatisticalphotoplethysmogramfeatures AT yunkyulee emotionrecognitionusingconvolutionalneuralnetworkwithselectedstatisticalphotoplethysmogramfeatures AT myotaeglim emotionrecognitionusingconvolutionalneuralnetworkwithselectedstatisticalphotoplethysmogramfeatures AT taekookang emotionrecognitionusingconvolutionalneuralnetworkwithselectedstatisticalphotoplethysmogramfeatures |