Understanding image representations by measuring their equivariance and equivalence

Despite the importance of image representations such as histograms of oriented gradients and deep Convolutional Neural Networks (CNN), our theoretical understanding of them remains limited. Aiming at filling this gap, we investigate three key mathematical properties of representations: equivariance,...

Ful tanımlama

Detaylı Bibliyografya
Asıl Yazarlar: Lenc, K, Vedaldi, A
Materyal Türü: Conference item
Baskı/Yayın Bilgisi: IEEE 2015
_version_ 1826297399760388096
author Lenc, K
Vedaldi, A
author_facet Lenc, K
Vedaldi, A
author_sort Lenc, K
collection OXFORD
description Despite the importance of image representations such as histograms of oriented gradients and deep Convolutional Neural Networks (CNN), our theoretical understanding of them remains limited. Aiming at filling this gap, we investigate three key mathematical properties of representations: equivariance, invariance, and equivalence. Equivariance studies how transformations of the input image are encoded by the representation, invariance being a special case where a transformation has no effect. Equivalence studies whether two representations, for example two different parametrisations of a CNN, capture the same visual information or not. A number of methods to establish these properties empirically are proposed, including introducing transformation and stitching layers in CNNs. These methods are then applied to popular representations to reveal insightful aspects of their structure, including clarifying at which layers in a CNN certain geometric invariances are achieved. While the focus of the paper is theoretical, direct applications to structured-output regression are demonstrated too.
first_indexed 2024-03-07T04:31:01Z
format Conference item
id oxford-uuid:ce52d423-ada8-4a57-8526-21ba12d2c3b9
institution University of Oxford
last_indexed 2024-03-07T04:31:01Z
publishDate 2015
publisher IEEE
record_format dspace
spelling oxford-uuid:ce52d423-ada8-4a57-8526-21ba12d2c3b92022-03-27T07:34:49ZUnderstanding image representations by measuring their equivariance and equivalenceConference itemhttp://purl.org/coar/resource_type/c_5794uuid:ce52d423-ada8-4a57-8526-21ba12d2c3b9Symplectic Elements at OxfordIEEE2015Lenc, KVedaldi, ADespite the importance of image representations such as histograms of oriented gradients and deep Convolutional Neural Networks (CNN), our theoretical understanding of them remains limited. Aiming at filling this gap, we investigate three key mathematical properties of representations: equivariance, invariance, and equivalence. Equivariance studies how transformations of the input image are encoded by the representation, invariance being a special case where a transformation has no effect. Equivalence studies whether two representations, for example two different parametrisations of a CNN, capture the same visual information or not. A number of methods to establish these properties empirically are proposed, including introducing transformation and stitching layers in CNNs. These methods are then applied to popular representations to reveal insightful aspects of their structure, including clarifying at which layers in a CNN certain geometric invariances are achieved. While the focus of the paper is theoretical, direct applications to structured-output regression are demonstrated too.
spellingShingle Lenc, K
Vedaldi, A
Understanding image representations by measuring their equivariance and equivalence
title Understanding image representations by measuring their equivariance and equivalence
title_full Understanding image representations by measuring their equivariance and equivalence
title_fullStr Understanding image representations by measuring their equivariance and equivalence
title_full_unstemmed Understanding image representations by measuring their equivariance and equivalence
title_short Understanding image representations by measuring their equivariance and equivalence
title_sort understanding image representations by measuring their equivariance and equivalence
work_keys_str_mv AT lenck understandingimagerepresentationsbymeasuringtheirequivarianceandequivalence
AT vedaldia understandingimagerepresentationsbymeasuringtheirequivarianceandequivalence