Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition
Face frontalization is the process of converting a face image under arbitrary pose to an image with frontal pose. Benefited from significant improvement of generative adversarial networks (GAN), generative models can use face frontalization to overcome the problem of model degradation owing to the v...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9098952/ |
| _version_ | 1830300712844656640 |
|---|---|
| author | Xiao Luan Hongmin Geng Linghui Liu Weisheng Li Yuanyuan Zhao Min Ren |
| author_facet | Xiao Luan Hongmin Geng Linghui Liu Weisheng Li Yuanyuan Zhao Min Ren |
| author_sort | Xiao Luan |
| collection | DOAJ |
| description | Face frontalization is the process of converting a face image under arbitrary pose to an image with frontal pose. Benefited from significant improvement of generative adversarial networks (GAN), generative models can use face frontalization to overcome the problem of model degradation owing to the variation of head pose in face recognition. Existing GAN based models can generate a synthesis face image with the same identity as the input, while those models are hard to capture the geometry structure or facial patterns via pixel-wise constraint, e.g. face contour. In this paper, we propose a Geometry Structure Preserving based GAN, i.e. GSP-GAN, for multi-pose face frontalization and recognition. The generator of our model takes the form of a typical auto-encoder, where the encoder extracts identity feature and the decoder synthesizes the corresponding frontal face image. In this process, the perception loss constrains the generator to synthesize a face image with the same identity as the input image. Meanwhile, we adopt real frontal face images as extra input data during training process, where a L1 norm loss is utilized to construct a pixel-wise mapping from arbitrary pose image to frontal image. More importantly, for discriminator of our model, we use the self-attention block to preserve the geometry structure of a face. The discriminator consists of a series of parallel sub-discriminators that carry the global and local attention information. Compared with the state-of-the-art models on datasets of Multi-PIE, LFW and CFP, the proposed GSP-GAN can generate high-quality frontal images under arbitrary pose, and get satisfactory recognition performance. |
| first_indexed | 2024-12-19T08:33:05Z |
| format | Article |
| id | doaj.art-c79d2b198818436699a6387cb6529cc8 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T08:33:05Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c79d2b198818436699a6387cb6529cc82022-12-21T20:29:07ZengIEEEIEEE Access2169-35362020-01-01810467610468710.1109/ACCESS.2020.29966379098952Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and RecognitionXiao Luan0https://orcid.org/0000-0003-0010-7361Hongmin Geng1https://orcid.org/0000-0003-0867-175XLinghui Liu2https://orcid.org/0000-0002-9849-5595Weisheng Li3https://orcid.org/0000-0002-9033-8245Yuanyuan Zhao4https://orcid.org/0000-0002-6678-9046Min Ren5https://orcid.org/0000-0002-3973-9713College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, ChinaCollege of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, ChinaSchool of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, ChinaCollege of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, ChinaCollege of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, ChinaCollege of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, ChinaFace frontalization is the process of converting a face image under arbitrary pose to an image with frontal pose. Benefited from significant improvement of generative adversarial networks (GAN), generative models can use face frontalization to overcome the problem of model degradation owing to the variation of head pose in face recognition. Existing GAN based models can generate a synthesis face image with the same identity as the input, while those models are hard to capture the geometry structure or facial patterns via pixel-wise constraint, e.g. face contour. In this paper, we propose a Geometry Structure Preserving based GAN, i.e. GSP-GAN, for multi-pose face frontalization and recognition. The generator of our model takes the form of a typical auto-encoder, where the encoder extracts identity feature and the decoder synthesizes the corresponding frontal face image. In this process, the perception loss constrains the generator to synthesize a face image with the same identity as the input image. Meanwhile, we adopt real frontal face images as extra input data during training process, where a L1 norm loss is utilized to construct a pixel-wise mapping from arbitrary pose image to frontal image. More importantly, for discriminator of our model, we use the self-attention block to preserve the geometry structure of a face. The discriminator consists of a series of parallel sub-discriminators that carry the global and local attention information. Compared with the state-of-the-art models on datasets of Multi-PIE, LFW and CFP, the proposed GSP-GAN can generate high-quality frontal images under arbitrary pose, and get satisfactory recognition performance.https://ieeexplore.ieee.org/document/9098952/Face frontalizationposegenerative adversarial networksself-attentiongeometry structure preserving |
| spellingShingle | Xiao Luan Hongmin Geng Linghui Liu Weisheng Li Yuanyuan Zhao Min Ren Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition IEEE Access Face frontalization pose generative adversarial networks self-attention geometry structure preserving |
| title | Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition |
| title_full | Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition |
| title_fullStr | Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition |
| title_full_unstemmed | Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition |
| title_short | Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition |
| title_sort | geometry structure preserving based gan for multi pose face frontalization and recognition |
| topic | Face frontalization pose generative adversarial networks self-attention geometry structure preserving |
| url | https://ieeexplore.ieee.org/document/9098952/ |
| work_keys_str_mv | AT xiaoluan geometrystructurepreservingbasedganformultiposefacefrontalizationandrecognition AT hongmingeng geometrystructurepreservingbasedganformultiposefacefrontalizationandrecognition AT linghuiliu geometrystructurepreservingbasedganformultiposefacefrontalizationandrecognition AT weishengli geometrystructurepreservingbasedganformultiposefacefrontalizationandrecognition AT yuanyuanzhao geometrystructurepreservingbasedganformultiposefacefrontalizationandrecognition AT minren geometrystructurepreservingbasedganformultiposefacefrontalizationandrecognition |