Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features
Object recognition, as one of the most fundamental and challenging problems in high-resolution remote sensing image interpretation, has received increasing attention in recent years. However, most conventional object recognition pipelines aim to recognize instances with bounding boxes in a supervise...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/13/3230 |
| _version_ | 1827598588153167872 |
|---|---|
| author | Yang Long Xiaofang Zhai Qiao Wan Xiaowei Tan |
| author_facet | Yang Long Xiaofang Zhai Qiao Wan Xiaowei Tan |
| author_sort | Yang Long |
| collection | DOAJ |
| description | Object recognition, as one of the most fundamental and challenging problems in high-resolution remote sensing image interpretation, has received increasing attention in recent years. However, most conventional object recognition pipelines aim to recognize instances with bounding boxes in a supervised learning strategy, which require intensive and manual labor for instance annotation creation. In this paper, we propose a weakly supervised learning method to alleviate this problem. The core idea of our method is to recognize multiple objects in an image using only image-level semantic labels and indicate the recognized objects with location points instead of box extent. Specifically, a deep convolutional neural network is first trained to perform semantic scene classification, of which the result is employed for the categorical determination of objects in an image. Then, by back-propagating the categorical feature from the fully connected layer to the deep convolutional layer, the categorical and spatial information of an image are combined to obtain an object discriminative localization map, which can effectively indicate the salient regions of objects. Next, a dynamic updating method of local response extremum is proposed to further determine the locations of objects in an image. Finally, extensive experiments are conducted to localize aircraft and oiltanks in remote sensing images based on different convolutional neural networks. Experimental results show that the proposed method outperforms the-state-of-the-art methods, achieving the precision, recall, and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>F</mi><mn>1</mn></msub></semantics></math></inline-formula>-score at 94.50%, 88.79%, and 91.56% for aircraft localization and 89.12%, 83.04%, and 85.97% for oiltank localization, respectively. We hope that our work could serve as a basic reference for remote sensing object localization via a weakly supervised strategy and provide new opportunities for further research. |
| first_indexed | 2024-03-09T03:55:10Z |
| format | Article |
| id | doaj.art-02f17d40e6984b39bce670666e719f22 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T03:55:10Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-02f17d40e6984b39bce670666e719f222023-12-03T14:21:25ZengMDPI AGRemote Sensing2072-42922022-07-011413323010.3390/rs14133230Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional FeaturesYang Long0Xiaofang Zhai1Qiao Wan2Xiaowei Tan3Guangzhou Urban Planning & Survey Design Research Institute, Guangzhou 510060, ChinaCollege of Urban and Environment Sciences, Hubei Normal University, Huangshi 435000, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, ChinaObject recognition, as one of the most fundamental and challenging problems in high-resolution remote sensing image interpretation, has received increasing attention in recent years. However, most conventional object recognition pipelines aim to recognize instances with bounding boxes in a supervised learning strategy, which require intensive and manual labor for instance annotation creation. In this paper, we propose a weakly supervised learning method to alleviate this problem. The core idea of our method is to recognize multiple objects in an image using only image-level semantic labels and indicate the recognized objects with location points instead of box extent. Specifically, a deep convolutional neural network is first trained to perform semantic scene classification, of which the result is employed for the categorical determination of objects in an image. Then, by back-propagating the categorical feature from the fully connected layer to the deep convolutional layer, the categorical and spatial information of an image are combined to obtain an object discriminative localization map, which can effectively indicate the salient regions of objects. Next, a dynamic updating method of local response extremum is proposed to further determine the locations of objects in an image. Finally, extensive experiments are conducted to localize aircraft and oiltanks in remote sensing images based on different convolutional neural networks. Experimental results show that the proposed method outperforms the-state-of-the-art methods, achieving the precision, recall, and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>F</mi><mn>1</mn></msub></semantics></math></inline-formula>-score at 94.50%, 88.79%, and 91.56% for aircraft localization and 89.12%, 83.04%, and 85.97% for oiltank localization, respectively. We hope that our work could serve as a basic reference for remote sensing object localization via a weakly supervised strategy and provide new opportunities for further research.https://www.mdpi.com/2072-4292/14/13/3230object localizationweakly supervised learning (WSL)deep convolutional featuresremote sensing images |
| spellingShingle | Yang Long Xiaofang Zhai Qiao Wan Xiaowei Tan Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features Remote Sensing object localization weakly supervised learning (WSL) deep convolutional features remote sensing images |
| title | Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features |
| title_full | Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features |
| title_fullStr | Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features |
| title_full_unstemmed | Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features |
| title_short | Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features |
| title_sort | object localization in weakly labeled remote sensing images based on deep convolutional features |
| topic | object localization weakly supervised learning (WSL) deep convolutional features remote sensing images |
| url | https://www.mdpi.com/2072-4292/14/13/3230 |
| work_keys_str_mv | AT yanglong objectlocalizationinweaklylabeledremotesensingimagesbasedondeepconvolutionalfeatures AT xiaofangzhai objectlocalizationinweaklylabeledremotesensingimagesbasedondeepconvolutionalfeatures AT qiaowan objectlocalizationinweaklylabeledremotesensingimagesbasedondeepconvolutionalfeatures AT xiaoweitan objectlocalizationinweaklylabeledremotesensingimagesbasedondeepconvolutionalfeatures |