DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension

We tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstructi...

Full description

Bibliographic Details
Main Authors: Shapovalov, R, Novotny, D, Graham, B, Labatut, P, Vedaldi, A
Format: Conference item
Language:English
Published: IEEE 2022
_version_ 1797067218344738816
author Shapovalov, R
Novotny, D
Graham, B
Labatut, P
Vedaldi, A
author_facet Shapovalov, R
Novotny, D
Graham, B
Labatut, P
Vedaldi, A
author_sort Shapovalov, R
collection OXFORD
description We tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstruction methods that use parametric models such as SMPL pre-trained on a large dataset of 3D object scans. Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species. The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object. The decomposition of the object into parts is regularized by expressing part assignments as a combination of the smooth eigenfunctions of the Laplace-Beltrami operator. We show significant improvements compared to state-of-the-art nonrigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals.
first_indexed 2024-03-06T21:53:14Z
format Conference item
id oxford-uuid:4c072d32-aad9-44ec-b41d-64074b5441c7
institution University of Oxford
language English
last_indexed 2024-03-06T21:53:14Z
publishDate 2022
publisher IEEE
record_format dspace
spelling oxford-uuid:4c072d32-aad9-44ec-b41d-64074b5441c72022-03-26T15:47:06ZDensePose 3D: lifting canonical surface maps of articulated objects to the third dimensionConference itemhttp://purl.org/coar/resource_type/c_5794uuid:4c072d32-aad9-44ec-b41d-64074b5441c7EnglishSymplectic ElementsIEEE2022Shapovalov, RNovotny, DGraham, BLabatut, PVedaldi, AWe tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. This is in stark contrast with previous deformable reconstruction methods that use parametric models such as SMPL pre-trained on a large dataset of 3D object scans. Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species. The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object. The decomposition of the object into parts is regularized by expressing part assignments as a combination of the smooth eigenfunctions of the Laplace-Beltrami operator. We show significant improvements compared to state-of-the-art nonrigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals.
spellingShingle Shapovalov, R
Novotny, D
Graham, B
Labatut, P
Vedaldi, A
DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
title DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
title_full DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
title_fullStr DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
title_full_unstemmed DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
title_short DensePose 3D: lifting canonical surface maps of articulated objects to the third dimension
title_sort densepose 3d lifting canonical surface maps of articulated objects to the third dimension
work_keys_str_mv AT shapovalovr densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension
AT novotnyd densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension
AT grahamb densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension
AT labatutp densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension
AT vedaldia densepose3dliftingcanonicalsurfacemapsofarticulatedobjectstothethirddimension