Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images
Weakly supervised object detection (WSOD) in remote sensing images (RSI) only require image-level labels to detect various objects. Most of the WSOD methods incline to capture the most discriminative parts of object rather than the entire object, and the number of easy and hard samples is imbalanced...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9712427/ |
| _version_ | 1828430071021436928 |
|---|---|
| author | Xiaoliang Qian Yu Huo Gong Cheng Xiwen Yao Ke Li Hangli Ren Wei Wang |
| author_facet | Xiaoliang Qian Yu Huo Gong Cheng Xiwen Yao Ke Li Hangli Ren Wei Wang |
| author_sort | Xiaoliang Qian |
| collection | DOAJ |
| description | Weakly supervised object detection (WSOD) in remote sensing images (RSI) only require image-level labels to detect various objects. Most of the WSOD methods incline to capture the most discriminative parts of object rather than the entire object, and the number of easy and hard samples is imbalanced. To address the first problem, a novel metric named objectness score (OS) is proposed and incorporated into the training loss of our WSOD model. The OS is consisted of the traditional class confidence score (CCS) and the object completeness prior score (OCPS). The CCS can provide the probability that a proposal belongs to a certain class, and the OCPS can quantify the completeness that a proposal covers the entire object. Therefore, the samples which cover the entire object with high class confidences will be assigned large weight in the training loss through OS. To handle the second problem, a novel metric named difficulty evaluation score (DES) is proposed and also incorporated into the training loss. The DES is calculated by using the entropy of confidence score vector of each proposal and is used to quantify how difficult a proposal can be identified correctly, consequently, the hard samples will also be assigned large weight in the training loss through DES. The ablation experiments on two RSI datasets verify the effectiveness of the proposed OS and DES. The comprehensive quantitative and subjective evaluations demonstrate that our method inclines to detect the entire object accurately, and surpasses seven state-of-the-art WSOD methods. |
| first_indexed | 2024-12-10T17:42:27Z |
| format | Article |
| id | doaj.art-935234cb9ea146a39392d2aa7c64f7cf |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-10T17:42:27Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-935234cb9ea146a39392d2aa7c64f7cf2022-12-22T01:39:19ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352022-01-01151902191110.1109/JSTARS.2022.31508439712427Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing ImagesXiaoliang Qian0https://orcid.org/0000-0002-4328-6411Yu Huo1Gong Cheng2https://orcid.org/0000-0001-5030-0683Xiwen Yao3https://orcid.org/0000-0002-7466-7428Ke Li4https://orcid.org/0000-0002-7873-1554Hangli Ren5Wei Wang6https://orcid.org/0000-0002-8770-3862College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaSchool of Automation, NorthWestern Polytechnical University, Xi’an, ChinaSchool of Automation, NorthWestern Polytechnical University, Xi’an, ChinaZhengzhou Institute of Surveying and Mapping, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaWeakly supervised object detection (WSOD) in remote sensing images (RSI) only require image-level labels to detect various objects. Most of the WSOD methods incline to capture the most discriminative parts of object rather than the entire object, and the number of easy and hard samples is imbalanced. To address the first problem, a novel metric named objectness score (OS) is proposed and incorporated into the training loss of our WSOD model. The OS is consisted of the traditional class confidence score (CCS) and the object completeness prior score (OCPS). The CCS can provide the probability that a proposal belongs to a certain class, and the OCPS can quantify the completeness that a proposal covers the entire object. Therefore, the samples which cover the entire object with high class confidences will be assigned large weight in the training loss through OS. To handle the second problem, a novel metric named difficulty evaluation score (DES) is proposed and also incorporated into the training loss. The DES is calculated by using the entropy of confidence score vector of each proposal and is used to quantify how difficult a proposal can be identified correctly, consequently, the hard samples will also be assigned large weight in the training loss through DES. The ablation experiments on two RSI datasets verify the effectiveness of the proposed OS and DES. The comprehensive quantitative and subjective evaluations demonstrate that our method inclines to detect the entire object accurately, and surpasses seven state-of-the-art WSOD methods.https://ieeexplore.ieee.org/document/9712427/Difficulty evaluation score (DES)object completeness prior scoreremote sensing image (RSI)weakly supervised object detection (WSOD) |
| spellingShingle | Xiaoliang Qian Yu Huo Gong Cheng Xiwen Yao Ke Li Hangli Ren Wei Wang Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Difficulty evaluation score (DES) object completeness prior score remote sensing image (RSI) weakly supervised object detection (WSOD) |
| title | Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images |
| title_full | Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images |
| title_fullStr | Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images |
| title_full_unstemmed | Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images |
| title_short | Incorporating the Completeness and Difficulty of Proposals Into Weakly Supervised Object Detection in Remote Sensing Images |
| title_sort | incorporating the completeness and difficulty of proposals into weakly supervised object detection in remote sensing images |
| topic | Difficulty evaluation score (DES) object completeness prior score remote sensing image (RSI) weakly supervised object detection (WSOD) |
| url | https://ieeexplore.ieee.org/document/9712427/ |
| work_keys_str_mv | AT xiaoliangqian incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages AT yuhuo incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages AT gongcheng incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages AT xiwenyao incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages AT keli incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages AT hangliren incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages AT weiwang incorporatingthecompletenessanddifficultyofproposalsintoweaklysupervisedobjectdetectioninremotesensingimages |