Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks
We propose an emotion recognition framework based on ResNet, bidirectional long- and short-term memory (BiLSTM) modules, and data augmentation using a ResNet deep convolutional generative adversarial network (DCGAN) with photoplethysmography (PPG) signals as input. The emotions identified in this st...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9947048/ |
| _version_ | 1811220141305757696 |
|---|---|
| author | Sung-Nien Yu Shao-Wei Wang Yu Ping Chang |
| author_facet | Sung-Nien Yu Shao-Wei Wang Yu Ping Chang |
| author_sort | Sung-Nien Yu |
| collection | DOAJ |
| description | We propose an emotion recognition framework based on ResNet, bidirectional long- and short-term memory (BiLSTM) modules, and data augmentation using a ResNet deep convolutional generative adversarial network (DCGAN) with photoplethysmography (PPG) signals as input. The emotions identified in this study were classified into two classes (positive and negative) and four classes (neutral, angry, happy, and sad). The framework achieved high recognition rates of 90.34% and 86.32% in two- and four-class emotion recognition tasks, respectively, outperforming other representative methods. Moreover, we show that the ResNet DCGAN module can synthesize samples that do not just look like those in the training set, but also capture discriminative features of the different classes. The distinguishability of the classes was enhanced when these synthetic samples were added to the original samples, which in turn improved the test accuracy of the model when trained using these mixed samples. This effect was evaluated using various quantitative and qualitative methods, including the inception score (IS), Fréchet inception distance (FID), GAN quality index (GQI), linear discriminant analysis (LDA), and Mahalanobis distance (MD). |
| first_indexed | 2024-04-12T07:36:36Z |
| format | Article |
| id | doaj.art-486ce4ab2b9945668a105763f180531f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T07:36:36Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-486ce4ab2b9945668a105763f180531f2022-12-22T03:41:55ZengIEEEIEEE Access2169-35362022-01-011011963011964010.1109/ACCESS.2022.32217749947048Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial NetworksSung-Nien Yu0https://orcid.org/0000-0001-7540-7233Shao-Wei Wang1Yu Ping Chang2https://orcid.org/0000-0001-7643-4233Department of Electrical Engineering, National Chung Cheng University, Minxiong, Chiayi County, TaiwanDepartment of Electrical Engineering, National Chung Cheng University, Minxiong, Chiayi County, TaiwanDepartment of Electrical Engineering, National Chung Cheng University, Minxiong, Chiayi County, TaiwanWe propose an emotion recognition framework based on ResNet, bidirectional long- and short-term memory (BiLSTM) modules, and data augmentation using a ResNet deep convolutional generative adversarial network (DCGAN) with photoplethysmography (PPG) signals as input. The emotions identified in this study were classified into two classes (positive and negative) and four classes (neutral, angry, happy, and sad). The framework achieved high recognition rates of 90.34% and 86.32% in two- and four-class emotion recognition tasks, respectively, outperforming other representative methods. Moreover, we show that the ResNet DCGAN module can synthesize samples that do not just look like those in the training set, but also capture discriminative features of the different classes. The distinguishability of the classes was enhanced when these synthetic samples were added to the original samples, which in turn improved the test accuracy of the model when trained using these mixed samples. This effect was evaluated using various quantitative and qualitative methods, including the inception score (IS), Fréchet inception distance (FID), GAN quality index (GQI), linear discriminant analysis (LDA), and Mahalanobis distance (MD).https://ieeexplore.ieee.org/document/9947048/AugmentationDCGANemotion recognitionPPG |
| spellingShingle | Sung-Nien Yu Shao-Wei Wang Yu Ping Chang Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks IEEE Access Augmentation DCGAN emotion recognition PPG |
| title | Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks |
| title_full | Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks |
| title_fullStr | Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks |
| title_full_unstemmed | Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks |
| title_short | Improving Distinguishability of Photoplethysmography in Emotion Recognition Using Deep Convolutional Generative Adversarial Networks |
| title_sort | improving distinguishability of photoplethysmography in emotion recognition using deep convolutional generative adversarial networks |
| topic | Augmentation DCGAN emotion recognition PPG |
| url | https://ieeexplore.ieee.org/document/9947048/ |
| work_keys_str_mv | AT sungnienyu improvingdistinguishabilityofphotoplethysmographyinemotionrecognitionusingdeepconvolutionalgenerativeadversarialnetworks AT shaoweiwang improvingdistinguishabilityofphotoplethysmographyinemotionrecognitionusingdeepconvolutionalgenerativeadversarialnetworks AT yupingchang improvingdistinguishabilityofphotoplethysmographyinemotionrecognitionusingdeepconvolutionalgenerativeadversarialnetworks |