DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
We tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstructi...
Main Authors: | , , , , |
---|---|
Format: | Conference item |
Language: | English |
Published: |
IEEE
2022
|
_version_ | 1797067218344738816 |
---|---|
author | Shapovalov, R Novotny, D Graham, B Labatut, P Vedaldi, A |
author_facet | Shapovalov, R Novotny, D Graham, B Labatut, P Vedaldi, A |
author_sort | Shapovalov, R |
collection | OXFORD |
description | We tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstruction methods that use parametric models such as SMPL pre-trained on a large dataset of 3D object scans. Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species. The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object. The decomposition of the object into parts is regularized by expressing part assignments as a combination of the smooth eigenfunctions of the Laplace-Beltrami operator. We show significant improvements compared to state-of-the-art nonrigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals. |
first_indexed | 2024-03-06T21:53:14Z |
format | Conference item |
id | oxford-uuid:4c072d32-aad9-44ec-b41d-64074b5441c7 |
institution | University of Oxford |
language | English |
last_indexed | 2024-03-06T21:53:14Z |
publishDate | 2022 |
publisher | IEEE |
record_format | dspace |
spelling | oxford-uuid:4c072d32-aad9-44ec-b41d-64074b5441c72022-03-26T15:47:06ZDensePose 3D: lifting canonical surface maps of articulated objects to the third dimensionConference itemhttp://purl.org/coar/resource_type/c_5794uuid:4c072d32-aad9-44ec-b41d-64074b5441c7EnglishSymplectic ElementsIEEE2022Shapovalov, RNovotny, DGraham, BLabatut, PVedaldi, AWe tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstruction methods that use parametric models such as SMPL pre-trained on a large dataset of 3D object scans. Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species. The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object. The decomposition of the object into parts is regularized by expressing part assignments as a combination of the smooth eigenfunctions of the Laplace-Beltrami operator. We show significant improvements compared to state-of-the-art nonrigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals. |
spellingShingle | Shapovalov, R Novotny, D Graham, B Labatut, P Vedaldi, A DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension |
title | DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension |
title_full | DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension |
title_fullStr | DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension |
title_full_unstemmed | DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension |
title_short | DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension |
title_sort | densepose 3d lifting canonical surface maps of articulated objects to the third dimension |
work_keys_str_mv | AT shapovalovr densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension AT novotnyd densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension AT grahamb densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension AT labatutp densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension AT vedaldia densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension |