Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement
Weakly supervised object detection (WSOD) aims to predict a set of bounding boxes and corresponding category labels for instances with only image-level supervisions. Compared with fully supervised object detection, WSOD in remote sensing images (RSIs) is much more challenging due to the vast foregro...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/7/1203 |
| _version_ | 1797211992014979072 |
|---|---|
| author | Shangdong Zheng Zebin Wu Yang Xu Zhihui Wei |
| author_facet | Shangdong Zheng Zebin Wu Yang Xu Zhihui Wei |
| author_sort | Shangdong Zheng |
| collection | DOAJ |
| description | Weakly supervised object detection (WSOD) aims to predict a set of bounding boxes and corresponding category labels for instances with only image-level supervisions. Compared with fully supervised object detection, WSOD in remote sensing images (RSIs) is much more challenging due to the vast foreground-related context regions. In this paper, we propose a progressive image-level and instance-level feature refinement network to address the problems of missing detection and part domination for WSOD in RSIs. Firstly, we propose a multi-label attention mining loss (MAML)-guided image-level feature refinement branch to effectively allocate the computational resources towards the most informative part of images. With the supervision of MAML, all latent instances in images are emphasized. However, image-level feature refinement further expands responsive gaps between the informative part and other sub-optimal informative ones, which results in exacerbating the problem of part domination. In order to alleviate the above-mentioned limitation, we further construct an instance-level feature refinement branch to re-balance the contributions of different adjacent candidate bounding boxes according to the detection task. An instance selection loss (ISL) is proposed to progressively boost the representation of salient regions by exploring supervision from the network itself. Finally, we integrate the image-level and instance-level feature refinement branches into a complete network and the proposed MAML and ISL functions are merged with class classification and box regression to optimize the whole WSOD network in an end-to-end training fashion. We conduct experiments on two popular WSOD datasets, NWPU VHR-10.v2 and DIOR. All the experimental results demonstrate that our method achieves a competitive performance compared with other state-of-the-art approaches. |
| first_indexed | 2024-04-24T10:35:17Z |
| format | Article |
| id | doaj.art-1fb058b0b4cf450fa8dcb410892213a8 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-04-24T10:35:17Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-1fb058b0b4cf450fa8dcb410892213a82024-04-12T13:25:37ZengMDPI AGRemote Sensing2072-42922024-03-01167120310.3390/rs16071203Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature RefinementShangdong Zheng0Zebin Wu1Yang Xu2Zhihui Wei3School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaWeakly supervised object detection (WSOD) aims to predict a set of bounding boxes and corresponding category labels for instances with only image-level supervisions. Compared with fully supervised object detection, WSOD in remote sensing images (RSIs) is much more challenging due to the vast foreground-related context regions. In this paper, we propose a progressive image-level and instance-level feature refinement network to address the problems of missing detection and part domination for WSOD in RSIs. Firstly, we propose a multi-label attention mining loss (MAML)-guided image-level feature refinement branch to effectively allocate the computational resources towards the most informative part of images. With the supervision of MAML, all latent instances in images are emphasized. However, image-level feature refinement further expands responsive gaps between the informative part and other sub-optimal informative ones, which results in exacerbating the problem of part domination. In order to alleviate the above-mentioned limitation, we further construct an instance-level feature refinement branch to re-balance the contributions of different adjacent candidate bounding boxes according to the detection task. An instance selection loss (ISL) is proposed to progressively boost the representation of salient regions by exploring supervision from the network itself. Finally, we integrate the image-level and instance-level feature refinement branches into a complete network and the proposed MAML and ISL functions are merged with class classification and box regression to optimize the whole WSOD network in an end-to-end training fashion. We conduct experiments on two popular WSOD datasets, NWPU VHR-10.v2 and DIOR. All the experimental results demonstrate that our method achieves a competitive performance compared with other state-of-the-art approaches.https://www.mdpi.com/2072-4292/16/7/1203weakly supervised learningremote sensing imagesobject detectionimage-level feature refinementinstance-level feature refinement |
| spellingShingle | Shangdong Zheng Zebin Wu Yang Xu Zhihui Wei Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement Remote Sensing weakly supervised learning remote sensing images object detection image-level feature refinement instance-level feature refinement |
| title | Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement |
| title_full | Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement |
| title_fullStr | Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement |
| title_full_unstemmed | Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement |
| title_short | Weakly Supervised Object Detection for Remote Sensing Images via Progressive Image-Level and Instance-Level Feature Refinement |
| title_sort | weakly supervised object detection for remote sensing images via progressive image level and instance level feature refinement |
| topic | weakly supervised learning remote sensing images object detection image-level feature refinement instance-level feature refinement |
| url | https://www.mdpi.com/2072-4292/16/7/1203 |
| work_keys_str_mv | AT shangdongzheng weaklysupervisedobjectdetectionforremotesensingimagesviaprogressiveimagelevelandinstancelevelfeaturerefinement AT zebinwu weaklysupervisedobjectdetectionforremotesensingimagesviaprogressiveimagelevelandinstancelevelfeaturerefinement AT yangxu weaklysupervisedobjectdetectionforremotesensingimagesviaprogressiveimagelevelandinstancelevelfeaturerefinement AT zhihuiwei weaklysupervisedobjectdetectionforremotesensingimagesviaprogressiveimagelevelandinstancelevelfeaturerefinement |