Object–Part Registration–Fusion Net for Fine-Grained Image Classification
Classifying fine-grained categories (e.g., bird species, car, and aircraft types) is a crucial problem in image understanding and is difficult due to intra-class and inter-class variance. Most of the existing fine-grained approaches individually utilize various parts and local information of objects...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Symmetry |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-8994/13/10/1838 |
| _version_ | 1797513112467800064 |
|---|---|
| author | Chih-Wei Lin Mengxiang Lin Jinfu Liu |
| author_facet | Chih-Wei Lin Mengxiang Lin Jinfu Liu |
| author_sort | Chih-Wei Lin |
| collection | DOAJ |
| description | Classifying fine-grained categories (e.g., bird species, car, and aircraft types) is a crucial problem in image understanding and is difficult due to intra-class and inter-class variance. Most of the existing fine-grained approaches individually utilize various parts and local information of objects to improve the classification accuracy but neglect the mechanism of the feature fusion between the object (global) and object’s parts (local) to reinforce fine-grained features. In this paper, we present a novel framework, namely object–part registration–fusion Net (OR-Net), which considers the mechanism of registration and fusion between an object (global) and its parts’ (local) features for fine-grained classification. Our model learns the fine-grained features from the object of global and local regions and fuses these features with the registration mechanism to reinforce each region’s characteristics in the feature maps. Precisely, OR-Net consists of: (1) a multi-stream feature extraction net, which generates features with global and various local regions of objects; (2) a registration–fusion feature module calculates the dimension and location relationships between global (object) regions and local (parts) regions to generate the registration information and fuses the local features into the global features with registration information to generate the fine-grained feature. Experiments execute symmetric GPU devices with symmetric mini-batch to verify that OR-Net surpasses the state-of-the-art approaches on CUB-200-2011 (Birds), Stanford-Cars, and Stanford-Aircraft datasets. |
| first_indexed | 2024-03-10T06:10:08Z |
| format | Article |
| id | doaj.art-175e4bf3a052469abfb9432a0c2fbbfd |
| institution | Directory Open Access Journal |
| issn | 2073-8994 |
| language | English |
| last_indexed | 2024-03-10T06:10:08Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Symmetry |
| spelling | doaj.art-175e4bf3a052469abfb9432a0c2fbbfd2023-11-22T20:09:51ZengMDPI AGSymmetry2073-89942021-10-011310183810.3390/sym13101838Object–Part Registration–Fusion Net for Fine-Grained Image ClassificationChih-Wei Lin0Mengxiang Lin1Jinfu Liu2College of Computer and Information Science, Fujian Agriculture and Forestry University, Fuzhou 350002, ChinaCollege of Computer and Information Science, Fujian Agriculture and Forestry University, Fuzhou 350002, ChinaCollege of Computer and Information Science, Fujian Agriculture and Forestry University, Fuzhou 350002, ChinaClassifying fine-grained categories (e.g., bird species, car, and aircraft types) is a crucial problem in image understanding and is difficult due to intra-class and inter-class variance. Most of the existing fine-grained approaches individually utilize various parts and local information of objects to improve the classification accuracy but neglect the mechanism of the feature fusion between the object (global) and object’s parts (local) to reinforce fine-grained features. In this paper, we present a novel framework, namely object–part registration–fusion Net (OR-Net), which considers the mechanism of registration and fusion between an object (global) and its parts’ (local) features for fine-grained classification. Our model learns the fine-grained features from the object of global and local regions and fuses these features with the registration mechanism to reinforce each region’s characteristics in the feature maps. Precisely, OR-Net consists of: (1) a multi-stream feature extraction net, which generates features with global and various local regions of objects; (2) a registration–fusion feature module calculates the dimension and location relationships between global (object) regions and local (parts) regions to generate the registration information and fuses the local features into the global features with registration information to generate the fine-grained feature. Experiments execute symmetric GPU devices with symmetric mini-batch to verify that OR-Net surpasses the state-of-the-art approaches on CUB-200-2011 (Birds), Stanford-Cars, and Stanford-Aircraft datasets.https://www.mdpi.com/2073-8994/13/10/1838fine-grained classificationconvolutional neural networkregistration |
| spellingShingle | Chih-Wei Lin Mengxiang Lin Jinfu Liu Object–Part Registration–Fusion Net for Fine-Grained Image Classification Symmetry fine-grained classification convolutional neural network registration |
| title | Object–Part Registration–Fusion Net for Fine-Grained Image Classification |
| title_full | Object–Part Registration–Fusion Net for Fine-Grained Image Classification |
| title_fullStr | Object–Part Registration–Fusion Net for Fine-Grained Image Classification |
| title_full_unstemmed | Object–Part Registration–Fusion Net for Fine-Grained Image Classification |
| title_short | Object–Part Registration–Fusion Net for Fine-Grained Image Classification |
| title_sort | object part registration fusion net for fine grained image classification |
| topic | fine-grained classification convolutional neural network registration |
| url | https://www.mdpi.com/2073-8994/13/10/1838 |
| work_keys_str_mv | AT chihweilin objectpartregistrationfusionnetforfinegrainedimageclassification AT mengxianglin objectpartregistrationfusionnetforfinegrainedimageclassification AT jinfuliu objectpartregistrationfusionnetforfinegrainedimageclassification |