DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers
In this paper, we address the few-shot classification task from a new perspective of optimal matching between image regions. We adopt the Earth Mover’s Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/144270 |
| _version_ | 1811696687378333696 |
|---|---|
| author | Zhang, Chi Cai, Yujun Lin, Guosheng Shen, Chunhua |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Zhang, Chi Cai, Yujun Lin, Guosheng Shen, Chunhua |
| author_sort | Zhang, Chi |
| collection | NTU |
| description | In this paper, we address the few-shot classification task from a new perspective of optimal matching between image regions. We adopt the Earth Mover’s Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the optimal matching flows between structural elements that have the minimum matching cost, which is used to represent the image distance for classification. To generate the important weights of elements in the EMD formulation, we design a cross-reference mechanism, which can effectively minimize the impact caused by the cluttered
background and large intra-class appearance variations. To handle k-shot classification, we propose to learn a structured fully connected layer that can directly classify dense image representations with the EMD. Based on the implicit
function theorem, the EMD can be inserted as a layer into the network for end-to-end training. We conduct comprehensive experiments to validate our algorithm and we set new state-of-the-art performance on four popular few-shot classification benchmarks, namely miniImageNet, tieredImageNet, Fewshot-CIFAR100 (FC100) and Caltech-UCSD Birds-200-2011 (CUB). |
| first_indexed | 2024-10-01T07:43:19Z |
| format | Conference Paper |
| id | ntu-10356/144270 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:43:19Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1442702020-10-26T06:04:12Z DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers Zhang, Chi Cai, Yujun Lin, Guosheng Shen, Chunhua School of Computer Science and Engineering IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020 Engineering::Computer science and engineering Deep Neural Networks Earth Mover’s Distance (EMD) In this paper, we address the few-shot classification task from a new perspective of optimal matching between image regions. We adopt the Earth Mover’s Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the optimal matching flows between structural elements that have the minimum matching cost, which is used to represent the image distance for classification. To generate the important weights of elements in the EMD formulation, we design a cross-reference mechanism, which can effectively minimize the impact caused by the cluttered background and large intra-class appearance variations. To handle k-shot classification, we propose to learn a structured fully connected layer that can directly classify dense image representations with the EMD. Based on the implicit function theorem, the EMD can be inserted as a layer into the network for end-to-end training. We conduct comprehensive experiments to validate our algorithm and we set new state-of-the-art performance on four popular few-shot classification benchmarks, namely miniImageNet, tieredImageNet, Fewshot-CIFAR100 (FC100) and Caltech-UCSD Birds-200-2011 (CUB). AI Singapore Ministry of Education (MOE) National Research Foundation (NRF) Accepted version This research is supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-RP-2018-003) and the MOE Tier-1 research grants: RG126/17 (S) and RG28/18 (S). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. This research is supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-RP-2018-003) and the MOE Tier-1 research grants: RG126/17 (S) and RG28/18 (S). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. 2020-10-26T06:04:12Z 2020-10-26T06:04:12Z 2020 Conference Paper Zhang, C., Cai, Y., Lin, G., & Shen, C. (2020). DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12203-12213. https://hdl.handle.net/10356/144270 12203 12213 en AISG-RP-2018-003 RG126/17 (S) RG28/18 (S) © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. application/pdf |
| spellingShingle | Engineering::Computer science and engineering Deep Neural Networks Earth Mover’s Distance (EMD) Zhang, Chi Cai, Yujun Lin, Guosheng Shen, Chunhua DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers |
| title | DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers |
| title_full | DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers |
| title_fullStr | DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers |
| title_full_unstemmed | DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers |
| title_short | DeepEMD : few-shot image classification with differentiable Earth Mover’s Distance and structured classifiers |
| title_sort | deepemd few shot image classification with differentiable earth mover s distance and structured classifiers |
| topic | Engineering::Computer science and engineering Deep Neural Networks Earth Mover’s Distance (EMD) |
| url | https://hdl.handle.net/10356/144270 |
| work_keys_str_mv | AT zhangchi deepemdfewshotimageclassificationwithdifferentiableearthmoversdistanceandstructuredclassifiers AT caiyujun deepemdfewshotimageclassificationwithdifferentiableearthmoversdistanceandstructuredclassifiers AT linguosheng deepemdfewshotimageclassificationwithdifferentiableearthmoversdistanceandstructuredclassifiers AT shenchunhua deepemdfewshotimageclassificationwithdifferentiableearthmoversdistanceandstructuredclassifiers |