Skeleton‐aware implicit function for single‐view human reconstruction
Abstract The aim is to reconstruct a complete and detailed clothed human from a single‐view input. Implicit function is suitable for this task because it represents fine shape details and varied topology. Current methods, however, often suffer from artefacts such as broken or disembodied body parts,...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-06-01
|
| Series: | CAAI Transactions on Intelligence Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cit2.12193 |
| _version_ | 1827126638473641984 |
|---|---|
| author | Pengpeng Liu Guixuan Zhang Shuwu Zhang Yuanhao Li Zhi Zeng |
| author_facet | Pengpeng Liu Guixuan Zhang Shuwu Zhang Yuanhao Li Zhi Zeng |
| author_sort | Pengpeng Liu |
| collection | DOAJ |
| description | Abstract The aim is to reconstruct a complete and detailed clothed human from a single‐view input. Implicit function is suitable for this task because it represents fine shape details and varied topology. Current methods, however, often suffer from artefacts such as broken or disembodied body parts, missing details, or depth ambiguity due to the ambiguity and complexity of human articulation. The main issue observed by the authors is structure‐agnostic. To address these problems, the authors fully utilise the skinned multi‐person linear (SMPL) model and propose a method using the Skeleton‐aware Implicit Function (SIF). To alleviate the broken or disembodied body parts, the proposed skeleton‐aware structure prior makes the skeleton awareness into an implicit function, which consists of a bone‐guided sampling strategy and a skeleton‐relative encoding strategy. To deal with the missing details and depth ambiguity problems, the authors’ body‐guided pixel‐aligned feature exploits the SMPL to enhance 2D normal and depth semantic features, and the proposed feature aggregation uses the extra geometry‐aware prior to enabling a more plausible merging with less noisy geometry. Additionally, SIF is also adapted to the RGB‐D input, and experimental results show that SIF outperforms the state‐of‐the‐arts methods on challenging datasets from Twindom and Thuman3.0. |
| first_indexed | 2024-03-11T13:45:53Z |
| format | Article |
| id | doaj.art-0d857e8b1d2e441eaea9a7400c7dfcc2 |
| institution | Directory Open Access Journal |
| issn | 2468-2322 |
| language | English |
| last_indexed | 2025-03-20T15:15:47Z |
| publishDate | 2023-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | CAAI Transactions on Intelligence Technology |
| spelling | doaj.art-0d857e8b1d2e441eaea9a7400c7dfcc22024-09-05T08:45:44ZengWileyCAAI Transactions on Intelligence Technology2468-23222023-06-018237938910.1049/cit2.12193Skeleton‐aware implicit function for single‐view human reconstructionPengpeng Liu0Guixuan Zhang1Shuwu Zhang2Yuanhao Li3Zhi Zeng4Key Laboratory of Digital Rights Services Institute of Automation Chinese Academy of Sciences Beijing ChinaKey Laboratory of Digital Rights Services Institute of Automation Chinese Academy of Sciences Beijing ChinaKey Laboratory of Digital Rights Services Institute of Automation Chinese Academy of Sciences Beijing ChinaInstitute of Innovative Research Tokyo Institute of Technology Yokohama JapanKey Laboratory of Digital Rights Services Institute of Automation Chinese Academy of Sciences Beijing ChinaAbstract The aim is to reconstruct a complete and detailed clothed human from a single‐view input. Implicit function is suitable for this task because it represents fine shape details and varied topology. Current methods, however, often suffer from artefacts such as broken or disembodied body parts, missing details, or depth ambiguity due to the ambiguity and complexity of human articulation. The main issue observed by the authors is structure‐agnostic. To address these problems, the authors fully utilise the skinned multi‐person linear (SMPL) model and propose a method using the Skeleton‐aware Implicit Function (SIF). To alleviate the broken or disembodied body parts, the proposed skeleton‐aware structure prior makes the skeleton awareness into an implicit function, which consists of a bone‐guided sampling strategy and a skeleton‐relative encoding strategy. To deal with the missing details and depth ambiguity problems, the authors’ body‐guided pixel‐aligned feature exploits the SMPL to enhance 2D normal and depth semantic features, and the proposed feature aggregation uses the extra geometry‐aware prior to enabling a more plausible merging with less noisy geometry. Additionally, SIF is also adapted to the RGB‐D input, and experimental results show that SIF outperforms the state‐of‐the‐arts methods on challenging datasets from Twindom and Thuman3.0.https://doi.org/10.1049/cit2.121933D human reconstructiondeep learningneural network |
| spellingShingle | Pengpeng Liu Guixuan Zhang Shuwu Zhang Yuanhao Li Zhi Zeng Skeleton‐aware implicit function for single‐view human reconstruction CAAI Transactions on Intelligence Technology 3D human reconstruction deep learning neural network |
| title | Skeleton‐aware implicit function for single‐view human reconstruction |
| title_full | Skeleton‐aware implicit function for single‐view human reconstruction |
| title_fullStr | Skeleton‐aware implicit function for single‐view human reconstruction |
| title_full_unstemmed | Skeleton‐aware implicit function for single‐view human reconstruction |
| title_short | Skeleton‐aware implicit function for single‐view human reconstruction |
| title_sort | skeleton aware implicit function for single view human reconstruction |
| topic | 3D human reconstruction deep learning neural network |
| url | https://doi.org/10.1049/cit2.12193 |
| work_keys_str_mv | AT pengpengliu skeletonawareimplicitfunctionforsingleviewhumanreconstruction AT guixuanzhang skeletonawareimplicitfunctionforsingleviewhumanreconstruction AT shuwuzhang skeletonawareimplicitfunctionforsingleviewhumanreconstruction AT yuanhaoli skeletonawareimplicitfunctionforsingleviewhumanreconstruction AT zhizeng skeletonawareimplicitfunctionforsingleviewhumanreconstruction |