Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning
Learning from limited labelled examples is key a research hotspot with excellent scenarios and potential applications. Currently, most of metric learning-based few-shot models still have the problem of low recognition accuracy. This is mainly because that they only use the top-layer abstract feature...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9093896/ |
| _version_ | 1819176177349165056 |
|---|---|
| author | Xiaoru Wang Bing Ma Zhihong Yu Fu Li Yali Cai |
| author_facet | Xiaoru Wang Bing Ma Zhihong Yu Fu Li Yali Cai |
| author_sort | Xiaoru Wang |
| collection | DOAJ |
| description | Learning from limited labelled examples is key a research hotspot with excellent scenarios and potential applications. Currently, most of metric learning-based few-shot models still have the problem of low recognition accuracy. This is mainly because that they only use the top-layer abstract feature with semantic information, which ignores the low-layer features that are also critical for the few-shot recognition. Therefore, the extracted features do not have abundant representation ability, and it is difficult to recognize easily confusing objects. Moreover, they usually adopt a fixed distance function or train a comparable network to measure features. These methods lack adaptability, cannot sufficiently fuse features, which leads to weaken the fitting ability of the metric function. And the same or different classes of images are treated equally, which makes the metric function have no emphasis point during training. To address these issues, we propose an end-to-end, metric learning-based model in this paper, called multi-scale decision network with feature fusion and weighting for few-shot learning (MSDN). Considering the importance of the low-layer features, we exploit a convolutional network to extract each layer feature. Then, we exploit a relation network to learn a non-linear metric between the support set and the query set features of each layer and classify the test images via a voting decision. During feature concatenation, we design a non-linear feature fusion item to improve the way of concatenation, so that the relation network can have a stronger function fitting ability to learn the relation score. Meanwhile, we introduce the attention mechanism by calculating the cosine similarity between the support set and the query set features as their weight, which makes the relation network pay more attention to the same class of images. Our model achieves the state-of-the-art accuracy result on Omniglot and miniImageNet datasets compared with popular few-shot recognition models. |
| first_indexed | 2024-12-22T21:06:37Z |
| format | Article |
| id | doaj.art-8114189208b0406b8bd6df3d5ec20292 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T21:06:37Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-8114189208b0406b8bd6df3d5ec202922022-12-21T18:12:39ZengIEEEIEEE Access2169-35362020-01-018921729218110.1109/ACCESS.2020.29948059093896Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot LearningXiaoru Wang0https://orcid.org/0000-0001-7171-2783Bing Ma1https://orcid.org/0000-0002-9856-3762Zhihong Yu2https://orcid.org/0000-0002-6379-1976Fu Li3https://orcid.org/0000-0001-8819-0547Yali Cai4https://orcid.org/0000-0002-8323-8182Beijing Key Laboratory of Network System and Network Culture, Beijing University of Posts and Telecommunications, Beijing, ChinaBeijing Key Laboratory of Network System and Network Culture, Beijing University of Posts and Telecommunications, Beijing, ChinaIntel China Research Center, Beijing, ChinaDepartment of Electrical and Computer Engineering, Portland State University, Portland, OR, USABeijing Key Laboratory of Network System and Network Culture, Beijing University of Posts and Telecommunications, Beijing, ChinaLearning from limited labelled examples is key a research hotspot with excellent scenarios and potential applications. Currently, most of metric learning-based few-shot models still have the problem of low recognition accuracy. This is mainly because that they only use the top-layer abstract feature with semantic information, which ignores the low-layer features that are also critical for the few-shot recognition. Therefore, the extracted features do not have abundant representation ability, and it is difficult to recognize easily confusing objects. Moreover, they usually adopt a fixed distance function or train a comparable network to measure features. These methods lack adaptability, cannot sufficiently fuse features, which leads to weaken the fitting ability of the metric function. And the same or different classes of images are treated equally, which makes the metric function have no emphasis point during training. To address these issues, we propose an end-to-end, metric learning-based model in this paper, called multi-scale decision network with feature fusion and weighting for few-shot learning (MSDN). Considering the importance of the low-layer features, we exploit a convolutional network to extract each layer feature. Then, we exploit a relation network to learn a non-linear metric between the support set and the query set features of each layer and classify the test images via a voting decision. During feature concatenation, we design a non-linear feature fusion item to improve the way of concatenation, so that the relation network can have a stronger function fitting ability to learn the relation score. Meanwhile, we introduce the attention mechanism by calculating the cosine similarity between the support set and the query set features as their weight, which makes the relation network pay more attention to the same class of images. Our model achieves the state-of-the-art accuracy result on Omniglot and miniImageNet datasets compared with popular few-shot recognition models.https://ieeexplore.ieee.org/document/9093896/Feature fusionfeature weightingfew-shot learningimage recognitionmulti-scale feature |
| spellingShingle | Xiaoru Wang Bing Ma Zhihong Yu Fu Li Yali Cai Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning IEEE Access Feature fusion feature weighting few-shot learning image recognition multi-scale feature |
| title | Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning |
| title_full | Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning |
| title_fullStr | Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning |
| title_full_unstemmed | Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning |
| title_short | Multi-Scale Decision Network With Feature Fusion and Weighting for Few-Shot Learning |
| title_sort | multi scale decision network with feature fusion and weighting for few shot learning |
| topic | Feature fusion feature weighting few-shot learning image recognition multi-scale feature |
| url | https://ieeexplore.ieee.org/document/9093896/ |
| work_keys_str_mv | AT xiaoruwang multiscaledecisionnetworkwithfeaturefusionandweightingforfewshotlearning AT bingma multiscaledecisionnetworkwithfeaturefusionandweightingforfewshotlearning AT zhihongyu multiscaledecisionnetworkwithfeaturefusionandweightingforfewshotlearning AT fuli multiscaledecisionnetworkwithfeaturefusionandweightingforfewshotlearning AT yalicai multiscaledecisionnetworkwithfeaturefusionandweightingforfewshotlearning |