Unsupervised learning of object landmarks through conditional image generation
We propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision. We cast this as the problem of generating images that combine the appearance of the object as seen in a first example image with the geometry of the object...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Published: |
Curran Associates
2018
|
| _version_ | 1797104111022243840 |
|---|---|
| author | Jakab, T Gupta, A Bilen, H Vedaldi, A |
| author_facet | Jakab, T Gupta, A Bilen, H Vedaldi, A |
| author_sort | Jakab, T |
| collection | OXFORD |
| description | We propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision. We cast this as the problem of generating images that combine the appearance of the object as seen in a first example image with the geometry of the object as seen in a second example image, where the two examples differ by a viewpoint change and/or an object deformation. In order to factorize appearance and geometry, we introduce a tight bottleneck in the geometry-extraction process that selects and distils geometryrelated features. Compared to standard image generation problems, which often use generative adversarial networks, our generation task is conditioned on both appearance and geometry and thus is significantly less ambiguous, to the point that adopting a simple perceptual loss formulation is sufficient. We demonstrate that our approach can learn object landmarks from synthetic image deformations or videos, all without manual supervision, while outperforming state-of-the-art unsupervised landmark detectors. We further show that our method is applicable to a large variety of datasets—faces, people, 3D objects, and digits—without any modifications. |
| first_indexed | 2024-03-07T06:29:12Z |
| format | Conference item |
| id | oxford-uuid:f562c769-774d-4d49-821d-2189bb213d00 |
| institution | University of Oxford |
| last_indexed | 2024-03-07T06:29:12Z |
| publishDate | 2018 |
| publisher | Curran Associates |
| record_format | dspace |
| spelling | oxford-uuid:f562c769-774d-4d49-821d-2189bb213d002022-03-27T12:27:04ZUnsupervised learning of object landmarks through conditional image generationConference itemhttp://purl.org/coar/resource_type/c_5794uuid:f562c769-774d-4d49-821d-2189bb213d00Symplectic Elements at OxfordCurran Associates2018Jakab, TGupta, ABilen, HVedaldi, AWe propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision. We cast this as the problem of generating images that combine the appearance of the object as seen in a first example image with the geometry of the object as seen in a second example image, where the two examples differ by a viewpoint change and/or an object deformation. In order to factorize appearance and geometry, we introduce a tight bottleneck in the geometry-extraction process that selects and distils geometryrelated features. Compared to standard image generation problems, which often use generative adversarial networks, our generation task is conditioned on both appearance and geometry and thus is significantly less ambiguous, to the point that adopting a simple perceptual loss formulation is sufficient. We demonstrate that our approach can learn object landmarks from synthetic image deformations or videos, all without manual supervision, while outperforming state-of-the-art unsupervised landmark detectors. We further show that our method is applicable to a large variety of datasets—faces, people, 3D objects, and digits—without any modifications. |
| spellingShingle | Jakab, T Gupta, A Bilen, H Vedaldi, A Unsupervised learning of object landmarks through conditional image generation |
| title | Unsupervised learning of object landmarks through conditional image generation |
| title_full | Unsupervised learning of object landmarks through conditional image generation |
| title_fullStr | Unsupervised learning of object landmarks through conditional image generation |
| title_full_unstemmed | Unsupervised learning of object landmarks through conditional image generation |
| title_short | Unsupervised learning of object landmarks through conditional image generation |
| title_sort | unsupervised learning of object landmarks through conditional image generation |
| work_keys_str_mv | AT jakabt unsupervisedlearningofobjectlandmarksthroughconditionalimagegeneration AT guptaa unsupervisedlearningofobjectlandmarksthroughconditionalimagegeneration AT bilenh unsupervisedlearningofobjectlandmarksthroughconditionalimagegeneration AT vedaldia unsupervisedlearningofobjectlandmarksthroughconditionalimagegeneration |