Transferring dense pose to proximal animal classes

Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail. In principle, the same approach could be extended to any animal class, but the effort required for collecting new annotations for each...

Full description

Bibliographic Details
Main Authors: Sanakoyeu, A, Khalidov, V, McCarthy, MS, Vedaldi, A, Neverova, N
Format: Conference item
Language:English
Published: IEEE 2020
_version_ 1797094975129780224
author Sanakoyeu, A
Khalidov, V
McCarthy, MS
Vedaldi, A
Neverova, N
author_facet Sanakoyeu, A
Khalidov, V
McCarthy, MS
Vedaldi, A
Neverova, N
author_sort Sanakoyeu, A
collection OXFORD
description Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail. In principle, the same approach could be extended to any animal class, but the effort required for collecting new annotations for each case makes this strategy impractical, despite important applications in natural conservation, science and business. We show that, at least for proximal animal classes such as chimpanzees, it is possible to transfer the knowledge existing in dense pose recognition for humans, as well as in more general object detectors and segmenters, to the problem of dense pose recognition in other classes. We do this by (1) establishing a DensePose model for the new animal which is also geometrically aligned to humans (2) introducing a multi-head R-CNN architecture that facilitates transfer of multiple recognition tasks between classes, (3) finding which combination of known classes can be transferred most effectively to the new animal and (4) using self-calibrated uncertainty heads to generate pseudo-labels graded by quality for training a model for this class. We also introduce two benchmark datasets labelled in the manner of DensePose for the class chimpanzee and use them to evaluate our approach, showing excellent transfer learning performance.
first_indexed 2024-03-07T04:21:23Z
format Conference item
id oxford-uuid:cb21b976-2341-4228-8b31-e4c1276f5f26
institution University of Oxford
language English
last_indexed 2024-03-07T04:21:23Z
publishDate 2020
publisher IEEE
record_format dspace
spelling oxford-uuid:cb21b976-2341-4228-8b31-e4c1276f5f262022-03-27T07:12:43ZTransferring dense pose to proximal animal classesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:cb21b976-2341-4228-8b31-e4c1276f5f26EnglishSymplectic ElementsIEEE2020Sanakoyeu, AKhalidov, VMcCarthy, MSVedaldi, ANeverova, NRecent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail. In principle, the same approach could be extended to any animal class, but the effort required for collecting new annotations for each case makes this strategy impractical, despite important applications in natural conservation, science and business. We show that, at least for proximal animal classes such as chimpanzees, it is possible to transfer the knowledge existing in dense pose recognition for humans, as well as in more general object detectors and segmenters, to the problem of dense pose recognition in other classes. We do this by (1) establishing a DensePose model for the new animal which is also geometrically aligned to humans (2) introducing a multi-head R-CNN architecture that facilitates transfer of multiple recognition tasks between classes, (3) finding which combination of known classes can be transferred most effectively to the new animal and (4) using self-calibrated uncertainty heads to generate pseudo-labels graded by quality for training a model for this class. We also introduce two benchmark datasets labelled in the manner of DensePose for the class chimpanzee and use them to evaluate our approach, showing excellent transfer learning performance.
spellingShingle Sanakoyeu, A
Khalidov, V
McCarthy, MS
Vedaldi, A
Neverova, N
Transferring dense pose to proximal animal classes
title Transferring dense pose to proximal animal classes
title_full Transferring dense pose to proximal animal classes
title_fullStr Transferring dense pose to proximal animal classes
title_full_unstemmed Transferring dense pose to proximal animal classes
title_short Transferring dense pose to proximal animal classes
title_sort transferring dense pose to proximal animal classes
work_keys_str_mv AT sanakoyeua transferringdenseposetoproximalanimalclasses
AT khalidovv transferringdenseposetoproximalanimalclasses
AT mccarthyms transferringdenseposetoproximalanimalclasses
AT vedaldia transferringdenseposetoproximalanimalclasses
AT neverovan transferringdenseposetoproximalanimalclasses