Unsupervised learning of landmarks by descriptor vector exchange
Equivariance to random image transformations is an effective method to learn landmarks of object categories, such as the eyes and the nose in faces, without manual supervision. However, this method does not explicitly guarantee that the learned landmarks are consistent with changes between different...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Jezik: | English |
| Izdano: |
IEEE
2020
|
| _version_ | 1826280631704748032 |
|---|---|
| author | Thewlis, J Albanie, S Bilen, H Vedaldi, A |
| author_facet | Thewlis, J Albanie, S Bilen, H Vedaldi, A |
| author_sort | Thewlis, J |
| collection | OXFORD |
| description | Equivariance to random image transformations is an effective method to learn landmarks of object categories, such as the eyes and the nose in faces, without manual supervision. However, this method does not explicitly guarantee that the learned landmarks are consistent with changes between different instances of the same object, such as different facial identities. In this paper, we develop a new perspective on the equivariance approach by noting that dense landmark detectors can be interpreted as local image descriptors equipped with invariance to intra-category variations. We then propose a direct method to enforce such an invariance in the standard equivariant loss. We do so by exchanging descriptor vectors between images of different object instances prior to matching them geometrically. In this manner, the same vectors must work regardless of the specific object identity considered. We use this approach to learn vectors that can simultaneously be interpreted as local descriptors and dense landmarks, combining the advantages of both. Experiments on standard benchmarks show that this approach can match, and in some cases surpass state-of-the-art performance amongst existing methods that learn landmarks without supervision. Code is available at www.robots.ox.ac.uk/~vgg/research/DVE/. |
| first_indexed | 2024-03-07T00:16:36Z |
| format | Conference item |
| id | oxford-uuid:7b0bd6f8-e7fb-4c84-86ec-35e5b9e762e4 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T00:16:36Z |
| publishDate | 2020 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:7b0bd6f8-e7fb-4c84-86ec-35e5b9e762e42022-03-26T20:48:07ZUnsupervised learning of landmarks by descriptor vector exchangeConference itemhttp://purl.org/coar/resource_type/c_5794uuid:7b0bd6f8-e7fb-4c84-86ec-35e5b9e762e4EnglishSymplectic Elements at OxfordIEEE2020Thewlis, JAlbanie, SBilen, HVedaldi, AEquivariance to random image transformations is an effective method to learn landmarks of object categories, such as the eyes and the nose in faces, without manual supervision. However, this method does not explicitly guarantee that the learned landmarks are consistent with changes between different instances of the same object, such as different facial identities. In this paper, we develop a new perspective on the equivariance approach by noting that dense landmark detectors can be interpreted as local image descriptors equipped with invariance to intra-category variations. We then propose a direct method to enforce such an invariance in the standard equivariant loss. We do so by exchanging descriptor vectors between images of different object instances prior to matching them geometrically. In this manner, the same vectors must work regardless of the specific object identity considered. We use this approach to learn vectors that can simultaneously be interpreted as local descriptors and dense landmarks, combining the advantages of both. Experiments on standard benchmarks show that this approach can match, and in some cases surpass state-of-the-art performance amongst existing methods that learn landmarks without supervision. Code is available at www.robots.ox.ac.uk/~vgg/research/DVE/. |
| spellingShingle | Thewlis, J Albanie, S Bilen, H Vedaldi, A Unsupervised learning of landmarks by descriptor vector exchange |
| title | Unsupervised learning of landmarks by descriptor vector exchange |
| title_full | Unsupervised learning of landmarks by descriptor vector exchange |
| title_fullStr | Unsupervised learning of landmarks by descriptor vector exchange |
| title_full_unstemmed | Unsupervised learning of landmarks by descriptor vector exchange |
| title_short | Unsupervised learning of landmarks by descriptor vector exchange |
| title_sort | unsupervised learning of landmarks by descriptor vector exchange |
| work_keys_str_mv | AT thewlisj unsupervisedlearningoflandmarksbydescriptorvectorexchange AT albanies unsupervisedlearningoflandmarksbydescriptorvectorexchange AT bilenh unsupervisedlearningoflandmarksbydescriptorvectorexchange AT vedaldia unsupervisedlearningoflandmarksbydescriptorvectorexchange |