Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization
Two of the biggest challenges in building models for detecting emotions from electroencephalography (EEG) devices are the relatively small amount of labeled samples and the strong variability of signal feature distributions between different subjects. In this study, we propose a context-generalized...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/5/1792 |
| _version_ | 1797414094309949440 |
|---|---|
| author | Juan Lorenzo Hagad Tsukasa Kimura Ken-ichi Fukui Masayuki Numao |
| author_facet | Juan Lorenzo Hagad Tsukasa Kimura Ken-ichi Fukui Masayuki Numao |
| author_sort | Juan Lorenzo Hagad |
| collection | DOAJ |
| description | Two of the biggest challenges in building models for detecting emotions from electroencephalography (EEG) devices are the relatively small amount of labeled samples and the strong variability of signal feature distributions between different subjects. In this study, we propose a context-generalized model that tackles the data constraints and subject variability simultaneously using a deep neural network architecture optimized for normally distributed subject-independent feature embeddings. Variational autoencoders (VAEs) at the input level allow the lower feature layers of the model to be trained on both labeled and unlabeled samples, maximizing the use of the limited data resources. Meanwhile, variational regularization encourages the model to learn Gaussian-distributed feature embeddings, resulting in robustness to small dataset imbalances. Subject-adversarial regularization applied to the bi-lateral features further enforces subject-independence on the final feature embedding used for emotion classification. The results from subject-independent performance experiments on the SEED and DEAP EEG-emotion datasets show that our model generalizes better across subjects than other state-of-the-art feature embeddings when paired with deep learning classifiers. Furthermore, qualitative analysis of the embedding space reveals that our proposed subject-invariant bi-lateral variational domain adversarial neural network (BiVDANN) architecture may improve the subject-independent performance by discovering normally distributed features. |
| first_indexed | 2024-03-09T05:28:08Z |
| format | Article |
| id | doaj.art-352fcd3dc2ea4aeb8f4d707a98f3a89c |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T05:28:08Z |
| publishDate | 2021-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-352fcd3dc2ea4aeb8f4d707a98f3a89c2023-12-03T12:35:03ZengMDPI AGSensors1424-82202021-03-01215179210.3390/s21051792Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial RegularizationJuan Lorenzo Hagad0Tsukasa Kimura1Ken-ichi Fukui2Masayuki Numao3Graduate School of Information Science and Technology, Osaka University, Suita, Osaka 565-0871, JapanInstitute of Scientific and Industrial Research, Osaka University, Ibaraki, Osaka 567-0047, JapanInstitute of Scientific and Industrial Research, Osaka University, Ibaraki, Osaka 567-0047, JapanInstitute of Scientific and Industrial Research, Osaka University, Ibaraki, Osaka 567-0047, JapanTwo of the biggest challenges in building models for detecting emotions from electroencephalography (EEG) devices are the relatively small amount of labeled samples and the strong variability of signal feature distributions between different subjects. In this study, we propose a context-generalized model that tackles the data constraints and subject variability simultaneously using a deep neural network architecture optimized for normally distributed subject-independent feature embeddings. Variational autoencoders (VAEs) at the input level allow the lower feature layers of the model to be trained on both labeled and unlabeled samples, maximizing the use of the limited data resources. Meanwhile, variational regularization encourages the model to learn Gaussian-distributed feature embeddings, resulting in robustness to small dataset imbalances. Subject-adversarial regularization applied to the bi-lateral features further enforces subject-independence on the final feature embedding used for emotion classification. The results from subject-independent performance experiments on the SEED and DEAP EEG-emotion datasets show that our model generalizes better across subjects than other state-of-the-art feature embeddings when paired with deep learning classifiers. Furthermore, qualitative analysis of the embedding space reveals that our proposed subject-invariant bi-lateral variational domain adversarial neural network (BiVDANN) architecture may improve the subject-independent performance by discovering normally distributed features.https://www.mdpi.com/1424-8220/21/5/1792electroencephalographyemotion modelingsubject independencedomain generalizationvariational autoencoderdomain adversarial network |
| spellingShingle | Juan Lorenzo Hagad Tsukasa Kimura Ken-ichi Fukui Masayuki Numao Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization Sensors electroencephalography emotion modeling subject independence domain generalization variational autoencoder domain adversarial network |
| title | Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization |
| title_full | Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization |
| title_fullStr | Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization |
| title_full_unstemmed | Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization |
| title_short | Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization |
| title_sort | learning subject generalized topographical eeg embeddings using deep variational autoencoders and domain adversarial regularization |
| topic | electroencephalography emotion modeling subject independence domain generalization variational autoencoder domain adversarial network |
| url | https://www.mdpi.com/1424-8220/21/5/1792 |
| work_keys_str_mv | AT juanlorenzohagad learningsubjectgeneralizedtopographicaleegembeddingsusingdeepvariationalautoencodersanddomainadversarialregularization AT tsukasakimura learningsubjectgeneralizedtopographicaleegembeddingsusingdeepvariationalautoencodersanddomainadversarialregularization AT kenichifukui learningsubjectgeneralizedtopographicaleegembeddingsusingdeepvariationalautoencodersanddomainadversarialregularization AT masayukinumao learningsubjectgeneralizedtopographicaleegembeddingsusingdeepvariationalautoencodersanddomainadversarialregularization |