Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition
Fine-grained image recognition is a highly challenging problem due to subtle differences between images. There are many attempts to solve fine-grained image recognition problems using data augmentation, jointly optimizing deep metric learning. CutMix is one of the excellent data augmentation strateg...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9313990/ |
| _version_ | 1828406430384783360 |
|---|---|
| author | Taehung Kim Hoseong Kim Hyeran Byun |
| author_facet | Taehung Kim Hoseong Kim Hyeran Byun |
| author_sort | Taehung Kim |
| collection | DOAJ |
| description | Fine-grained image recognition is a highly challenging problem due to subtle differences between images. There are many attempts to solve fine-grained image recognition problems using data augmentation, jointly optimizing deep metric learning. CutMix is one of the excellent data augmentation strategies which crops and merges to generate new images. However, it sometimes generates meaningless and obscured object images that degrade recognition performance. We propose a novel framework that solves the above problem and expands the CutMix leveraging localizing method. Also, we improve the recognition accuracy to joint optimizing with a pairwise margin loss using generated images from the improved CutMix. There are some images similar to the reference image among the generated images. They are generated by replacing similar parts from the reference image. Those generated images should not be located much farther than the margin value in embedding space because those generated images and a reference image have similar semantic meaning. However, the conventional margin loss can not consider those images which are located much farther than the margin. To solve this problem, we propose an additional margin loss to consider those generated images. The proposed framework consists of two stages: the part localization-aware CutMix and an adaptive pairwise margin loss. The proposed method achieves state-of-the-art performance on the CUB-200-2011, FGVC-Aircraft, Stanford Cars, and DeepFashion datasets. Furthermore, extensive experiments demonstrate that each stage improves the final performance. |
| first_indexed | 2024-12-10T11:11:36Z |
| format | Article |
| id | doaj.art-b0214248190c49e8b6d81550df48126f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T11:11:36Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b0214248190c49e8b6d81550df48126f2022-12-22T01:51:24ZengIEEEIEEE Access2169-35362021-01-0198786879610.1109/ACCESS.2021.30493059313990Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image RecognitionTaehung Kim0https://orcid.org/0000-0002-2233-7881Hoseong Kim1https://orcid.org/0000-0002-1583-1558Hyeran Byun2https://orcid.org/0000-0002-3082-3214Department of Computer Science, Yonsei University, Seoul, South KoreaAgency for Defense Development, Daejeon, South KoreaDepartment of Computer Science, Yonsei University, Seoul, South KoreaFine-grained image recognition is a highly challenging problem due to subtle differences between images. There are many attempts to solve fine-grained image recognition problems using data augmentation, jointly optimizing deep metric learning. CutMix is one of the excellent data augmentation strategies which crops and merges to generate new images. However, it sometimes generates meaningless and obscured object images that degrade recognition performance. We propose a novel framework that solves the above problem and expands the CutMix leveraging localizing method. Also, we improve the recognition accuracy to joint optimizing with a pairwise margin loss using generated images from the improved CutMix. There are some images similar to the reference image among the generated images. They are generated by replacing similar parts from the reference image. Those generated images should not be located much farther than the margin value in embedding space because those generated images and a reference image have similar semantic meaning. However, the conventional margin loss can not consider those images which are located much farther than the margin. To solve this problem, we propose an additional margin loss to consider those generated images. The proposed framework consists of two stages: the part localization-aware CutMix and an adaptive pairwise margin loss. The proposed method achieves state-of-the-art performance on the CUB-200-2011, FGVC-Aircraft, Stanford Cars, and DeepFashion datasets. Furthermore, extensive experiments demonstrate that each stage improves the final performance.https://ieeexplore.ieee.org/document/9313990/Adaptive margindeep neural networksfine-grained image recognitionmetric learningimage augmentationimage generation |
| spellingShingle | Taehung Kim Hoseong Kim Hyeran Byun Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition IEEE Access Adaptive margin deep neural networks fine-grained image recognition metric learning image augmentation image generation |
| title | Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition |
| title_full | Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition |
| title_fullStr | Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition |
| title_full_unstemmed | Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition |
| title_short | Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition |
| title_sort | localization aware adaptive pairwise margin loss for fine grained image recognition |
| topic | Adaptive margin deep neural networks fine-grained image recognition metric learning image augmentation image generation |
| url | https://ieeexplore.ieee.org/document/9313990/ |
| work_keys_str_mv | AT taehungkim localizationawareadaptivepairwisemarginlossforfinegrainedimagerecognition AT hoseongkim localizationawareadaptivepairwisemarginlossforfinegrainedimagerecognition AT hyeranbyun localizationawareadaptivepairwisemarginlossforfinegrainedimagerecognition |