Full Face-and-Head 3D Model With Photorealistic Texture
In the recent period, significant progress has been achieved towards reconstructing the 3D face model from face image. With the support of the render engines and sufficient data, the reconstruction results are fine in detail. Nevertheless, the research on the 3D face reconstruction with texture from...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9241395/ |
| _version_ | 1818351298409398272 |
|---|---|
| author | Yangyu Fan Yang Liu Guoyun Lv Shiya Liu Gen Li Yanhui Huang |
| author_facet | Yangyu Fan Yang Liu Guoyun Lv Shiya Liu Gen Li Yanhui Huang |
| author_sort | Yangyu Fan |
| collection | DOAJ |
| description | In the recent period, significant progress has been achieved towards reconstructing the 3D face model from face image. With the support of the render engines and sufficient data, the reconstruction results are fine in detail. Nevertheless, the research on the 3D face reconstruction with texture from a single unrestricted face image is imperfect. The rebuild process lacks essential structure and texture information in the profile and the craniofacial region. To address this problem, we present a method of creating a 3D full face-and-head model with photorealistic texture from a single “in-the-wild” face image in this paper. To this end, we introduce a pipeline to integrate the highly-detailed face model into the basic model. Specifically, the basic model was built by multilinear optimization, and the highly-detailed face model which represents the facial features generated by constrained illumination distribution. Additionally, to infer the invisible region texture information corresponding to the input face image, we design an effective architecture with the generative adversarial network (GAN) for panoramic UV texture generation. The final results after UV texture mapping were visualized in the experiment, which demonstrates that the model faithfully recovers the photorealistic details in arbitrary perspective. Furthermore, compared to the state-of-the-art facial modeling techniques and existing commercial solutions, our method takes less input and performs better in surface detail. |
| first_indexed | 2024-12-13T18:35:31Z |
| format | Article |
| id | doaj.art-c75ff03b8878453ba3db0290e78f1a1a |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T18:35:31Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c75ff03b8878453ba3db0290e78f1a1a2022-12-21T23:35:22ZengIEEEIEEE Access2169-35362020-01-01821070921072110.1109/ACCESS.2020.30318869241395Full Face-and-Head 3D Model With Photorealistic TextureYangyu Fan0https://orcid.org/0000-0003-0689-5418Yang Liu1https://orcid.org/0000-0001-5971-1040Guoyun Lv2https://orcid.org/0000-0002-8330-1911Shiya Liu3https://orcid.org/0000-0002-0245-9961Gen Li4Yanhui Huang5School of Electronics and Information, Northwestern Polytechnical University, Xi’an, ChinaContent Production Center of Virtual Reality, Beijing, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an, ChinaContent Production Center of Virtual Reality, Beijing, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an, ChinaIn the recent period, significant progress has been achieved towards reconstructing the 3D face model from face image. With the support of the render engines and sufficient data, the reconstruction results are fine in detail. Nevertheless, the research on the 3D face reconstruction with texture from a single unrestricted face image is imperfect. The rebuild process lacks essential structure and texture information in the profile and the craniofacial region. To address this problem, we present a method of creating a 3D full face-and-head model with photorealistic texture from a single “in-the-wild” face image in this paper. To this end, we introduce a pipeline to integrate the highly-detailed face model into the basic model. Specifically, the basic model was built by multilinear optimization, and the highly-detailed face model which represents the facial features generated by constrained illumination distribution. Additionally, to infer the invisible region texture information corresponding to the input face image, we design an effective architecture with the generative adversarial network (GAN) for panoramic UV texture generation. The final results after UV texture mapping were visualized in the experiment, which demonstrates that the model faithfully recovers the photorealistic details in arbitrary perspective. Furthermore, compared to the state-of-the-art facial modeling techniques and existing commercial solutions, our method takes less input and performs better in surface detail.https://ieeexplore.ieee.org/document/9241395/3D face reconstructionfull face-and-head model3D morphable modelUV texturegenerative adversarial networks |
| spellingShingle | Yangyu Fan Yang Liu Guoyun Lv Shiya Liu Gen Li Yanhui Huang Full Face-and-Head 3D Model With Photorealistic Texture IEEE Access 3D face reconstruction full face-and-head model 3D morphable model UV texture generative adversarial networks |
| title | Full Face-and-Head 3D Model With Photorealistic Texture |
| title_full | Full Face-and-Head 3D Model With Photorealistic Texture |
| title_fullStr | Full Face-and-Head 3D Model With Photorealistic Texture |
| title_full_unstemmed | Full Face-and-Head 3D Model With Photorealistic Texture |
| title_short | Full Face-and-Head 3D Model With Photorealistic Texture |
| title_sort | full face and head 3d model with photorealistic texture |
| topic | 3D face reconstruction full face-and-head model 3D morphable model UV texture generative adversarial networks |
| url | https://ieeexplore.ieee.org/document/9241395/ |
| work_keys_str_mv | AT yangyufan fullfaceandhead3dmodelwithphotorealistictexture AT yangliu fullfaceandhead3dmodelwithphotorealistictexture AT guoyunlv fullfaceandhead3dmodelwithphotorealistictexture AT shiyaliu fullfaceandhead3dmodelwithphotorealistictexture AT genli fullfaceandhead3dmodelwithphotorealistictexture AT yanhuihuang fullfaceandhead3dmodelwithphotorealistictexture |