EC-GAN: Emotion-Controllable GAN for Face Image Completion
Image completion methods based on deep learning, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have succeeded in producing semantically plausible results. However, existing facial image completion methods can either produce only one result or, although they can...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/13/7638 |
| _version_ | 1797592133919571968 |
|---|---|
| author | Yueqiao Chen Wenxia Yang Xi Fang Huan Han |
| author_facet | Yueqiao Chen Wenxia Yang Xi Fang Huan Han |
| author_sort | Yueqiao Chen |
| collection | DOAJ |
| description | Image completion methods based on deep learning, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have succeeded in producing semantically plausible results. However, existing facial image completion methods can either produce only one result or, although they can provide multiple results, cannot attribute particular emotions to the results. We propose EC-GAN, a novel facial Emotion-Controllable GAN-based image completion model that can infer and customize generative facial emotions. We propose an emotion inference module that infers the emotions of faces based on the unmasked regions of the faces. The emotion inference module is trained in a supervised manner and enforces the encoder to disentangle the emotion semantics from the native latent space. We also developed an emotion control module to modify the latent codes of emotions, moving the latent codes of the initial emotion toward the desired one while maintaining the remaining facial features. Extensive experiments were conducted on two facial datasets, CelebA-HQ and CFEED. Quantitative and qualitative results indicate that EC-GAN produces images with diverse desired expressions even when the main features of the faces are masked. On the other hand, EC-GAN promotes semantic inference capability with irregularly masked holes, resulting in more natural facial expressions. |
| first_indexed | 2024-03-11T01:47:09Z |
| format | Article |
| id | doaj.art-7f6570922d7e4b4aa59a4d523a8ee392 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T01:47:09Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-7f6570922d7e4b4aa59a4d523a8ee3922023-11-18T16:09:26ZengMDPI AGApplied Sciences2076-34172023-06-011313763810.3390/app13137638EC-GAN: Emotion-Controllable GAN for Face Image CompletionYueqiao Chen0Wenxia Yang1Xi Fang2Huan Han3School of Mathematics, Wuhan University of Technology, Wuhan 430070, ChinaSchool of Mathematics, Wuhan University of Technology, Wuhan 430070, ChinaSchool of Mathematics, Wuhan University of Technology, Wuhan 430070, ChinaSchool of Mathematics, Wuhan University of Technology, Wuhan 430070, ChinaImage completion methods based on deep learning, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have succeeded in producing semantically plausible results. However, existing facial image completion methods can either produce only one result or, although they can provide multiple results, cannot attribute particular emotions to the results. We propose EC-GAN, a novel facial Emotion-Controllable GAN-based image completion model that can infer and customize generative facial emotions. We propose an emotion inference module that infers the emotions of faces based on the unmasked regions of the faces. The emotion inference module is trained in a supervised manner and enforces the encoder to disentangle the emotion semantics from the native latent space. We also developed an emotion control module to modify the latent codes of emotions, moving the latent codes of the initial emotion toward the desired one while maintaining the remaining facial features. Extensive experiments were conducted on two facial datasets, CelebA-HQ and CFEED. Quantitative and qualitative results indicate that EC-GAN produces images with diverse desired expressions even when the main features of the faces are masked. On the other hand, EC-GAN promotes semantic inference capability with irregularly masked holes, resulting in more natural facial expressions.https://www.mdpi.com/2076-3417/13/13/7638image completiongenerative adversarial networkconditional variational autoencoderfacial emotion |
| spellingShingle | Yueqiao Chen Wenxia Yang Xi Fang Huan Han EC-GAN: Emotion-Controllable GAN for Face Image Completion Applied Sciences image completion generative adversarial network conditional variational autoencoder facial emotion |
| title | EC-GAN: Emotion-Controllable GAN for Face Image Completion |
| title_full | EC-GAN: Emotion-Controllable GAN for Face Image Completion |
| title_fullStr | EC-GAN: Emotion-Controllable GAN for Face Image Completion |
| title_full_unstemmed | EC-GAN: Emotion-Controllable GAN for Face Image Completion |
| title_short | EC-GAN: Emotion-Controllable GAN for Face Image Completion |
| title_sort | ec gan emotion controllable gan for face image completion |
| topic | image completion generative adversarial network conditional variational autoencoder facial emotion |
| url | https://www.mdpi.com/2076-3417/13/13/7638 |
| work_keys_str_mv | AT yueqiaochen ecganemotioncontrollableganforfaceimagecompletion AT wenxiayang ecganemotioncontrollableganforfaceimagecompletion AT xifang ecganemotioncontrollableganforfaceimagecompletion AT huanhan ecganemotioncontrollableganforfaceimagecompletion |