Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model
Face reconstruction is a popular topic in 3D vision system. However, traditional methods often depend on monocular cues, which contain few feature pixels and only use their location information while ignoring a lot of textural information. Furthermore, they are affected by the accuracy of the featur...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/3/459 |
| _version_ | 1828391850784849920 |
|---|---|
| author | Liang Tian Jing Liu Wei Guo |
| author_facet | Liang Tian Jing Liu Wei Guo |
| author_sort | Liang Tian |
| collection | DOAJ |
| description | Face reconstruction is a popular topic in 3D vision system. However, traditional methods often depend on monocular cues, which contain few feature pixels and only use their location information while ignoring a lot of textural information. Furthermore, they are affected by the accuracy of the feature extraction method and occlusion. Here, we propose a novel facial reconstruction framework that accurately extracts the 3D shapes and poses of faces from images captured at multi-views. It extends the traditional method using the monocular bilinear model to the multi-view-based bilinear model by incorporating the feature prior constraint and the texture constraint, which are learned from multi-view images. The feature prior constraint is used as a shape prior to allowing us to estimate accurate 3D facial contours. Furthermore, the texture constraint extracts a high-precision 3D facial shape where traditional methods fail because of their limited number of feature points or the mostly texture-less and texture-repetitive nature of the input images. Meanwhile, it fully explores the implied 3D information of the multi-view images, which also enhances the robustness of the results. Additionally, the proposed method uses only two or more uncalibrated images with an arbitrary baseline, estimating calibration and shape simultaneously. A comparison with the state-of-the-art monocular bilinear model-based method shows that the proposed method has a significantly higher level of accuracy. |
| first_indexed | 2024-12-10T07:10:58Z |
| format | Article |
| id | doaj.art-4e94f695d5ee4746afe18e3b98f015b2 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-12-10T07:10:58Z |
| publishDate | 2019-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-4e94f695d5ee4746afe18e3b98f015b22022-12-22T01:58:04ZengMDPI AGSensors1424-82202019-01-0119345910.3390/s19030459s19030459Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear ModelLiang Tian0Jing Liu1Wei Guo2The Key Laboratory of Augmented Reality, College of Mathematics and Information Science, Hebei Normal University, No.20 Road East, 2nd Ring South, Yuhua District, Shijiazhuang 050024, Hebei, ChinaThe Key Laboratory of Augmented Reality, College of Mathematics and Information Science, Hebei Normal University, No.20 Road East, 2nd Ring South, Yuhua District, Shijiazhuang 050024, Hebei, ChinaThe Key Laboratory of Augmented Reality, College of Mathematics and Information Science, Hebei Normal University, No.20 Road East, 2nd Ring South, Yuhua District, Shijiazhuang 050024, Hebei, ChinaFace reconstruction is a popular topic in 3D vision system. However, traditional methods often depend on monocular cues, which contain few feature pixels and only use their location information while ignoring a lot of textural information. Furthermore, they are affected by the accuracy of the feature extraction method and occlusion. Here, we propose a novel facial reconstruction framework that accurately extracts the 3D shapes and poses of faces from images captured at multi-views. It extends the traditional method using the monocular bilinear model to the multi-view-based bilinear model by incorporating the feature prior constraint and the texture constraint, which are learned from multi-view images. The feature prior constraint is used as a shape prior to allowing us to estimate accurate 3D facial contours. Furthermore, the texture constraint extracts a high-precision 3D facial shape where traditional methods fail because of their limited number of feature points or the mostly texture-less and texture-repetitive nature of the input images. Meanwhile, it fully explores the implied 3D information of the multi-view images, which also enhances the robustness of the results. Additionally, the proposed method uses only two or more uncalibrated images with an arbitrary baseline, estimating calibration and shape simultaneously. A comparison with the state-of-the-art monocular bilinear model-based method shows that the proposed method has a significantly higher level of accuracy.https://www.mdpi.com/1424-8220/19/3/4593D reconstruction3D visionmulti-view-based bilinear modelmodel matching3D shape modeling |
| spellingShingle | Liang Tian Jing Liu Wei Guo Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model Sensors 3D reconstruction 3D vision multi-view-based bilinear model model matching 3D shape modeling |
| title | Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model |
| title_full | Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model |
| title_fullStr | Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model |
| title_full_unstemmed | Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model |
| title_short | Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model |
| title_sort | three dimensional face reconstruction using multi view based bilinear model |
| topic | 3D reconstruction 3D vision multi-view-based bilinear model model matching 3D shape modeling |
| url | https://www.mdpi.com/1424-8220/19/3/459 |
| work_keys_str_mv | AT liangtian threedimensionalfacereconstructionusingmultiviewbasedbilinearmodel AT jingliu threedimensionalfacereconstructionusingmultiviewbasedbilinearmodel AT weiguo threedimensionalfacereconstructionusingmultiviewbasedbilinearmodel |