Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions
We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently int...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10233007/ |
| _version_ | 1797656252646424576 |
|---|---|
| author | Egor Burkov Ruslan Rakhimov Aleksandr Safin Evgeny Burnaev Victor Lempitsky |
| author_facet | Egor Burkov Ruslan Rakhimov Aleksandr Safin Evgeny Burnaev Victor Lempitsky |
| author_sort | Egor Burkov |
| collection | DOAJ |
| description | We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation– neural implicit functions– which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results. |
| first_indexed | 2024-03-11T17:26:40Z |
| format | Article |
| id | doaj.art-6ebc59c628a6465dae4803e736f52dbf |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T17:26:40Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-6ebc59c628a6465dae4803e736f52dbf2023-10-18T23:00:28ZengIEEEIEEE Access2169-35362023-01-0111956819569110.1109/ACCESS.2023.330941210233007Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit FunctionsEgor Burkov0https://orcid.org/0000-0003-2072-8093Ruslan Rakhimov1https://orcid.org/0000-0002-6763-3510Aleksandr Safin2https://orcid.org/0000-0002-5453-1101Evgeny Burnaev3https://orcid.org/0000-0001-8424-0690Victor Lempitsky4https://orcid.org/0000-0003-4118-710XSkolkovo Institute of Science and Technology, Moscow, RussiaSkolkovo Institute of Science and Technology, Moscow, RussiaSkolkovo Institute of Science and Technology, Moscow, RussiaSkolkovo Institute of Science and Technology, Moscow, RussiaIndependent Researcher, Edinburgh, U.KWe present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation– neural implicit functions– which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.https://ieeexplore.ieee.org/document/10233007/3D portraits3D reconstructionfew-shothead reconstructionmeta-learningneural implicit functions |
| spellingShingle | Egor Burkov Ruslan Rakhimov Aleksandr Safin Evgeny Burnaev Victor Lempitsky Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions IEEE Access 3D portraits 3D reconstruction few-shot head reconstruction meta-learning neural implicit functions |
| title | Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions |
| title_full | Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions |
| title_fullStr | Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions |
| title_full_unstemmed | Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions |
| title_short | Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions |
| title_sort | multi neus 3d head portraits from single image with neural implicit functions |
| topic | 3D portraits 3D reconstruction few-shot head reconstruction meta-learning neural implicit functions |
| url | https://ieeexplore.ieee.org/document/10233007/ |
| work_keys_str_mv | AT egorburkov multineus3dheadportraitsfromsingleimagewithneuralimplicitfunctions AT ruslanrakhimov multineus3dheadportraitsfromsingleimagewithneuralimplicitfunctions AT aleksandrsafin multineus3dheadportraitsfromsingleimagewithneuralimplicitfunctions AT evgenyburnaev multineus3dheadportraitsfromsingleimagewithneuralimplicitfunctions AT victorlempitsky multineus3dheadportraitsfromsingleimagewithneuralimplicitfunctions |