Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images
Weakly supervised object detection (WSOD) has a great practical value in remote sensing image (RSI) interpretation because the instance-level annotations are not required. The multiple instance learning based methods are mainstream, and two problems should be addressed. First of all, the majority of...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10214659/ |
| _version_ | 1797739744866598912 |
|---|---|
| author | Xiaoliang Qian Chenhao Wang Chao Li Zhehui Li Li Zeng Wei Wang QingE Wu |
| author_facet | Xiaoliang Qian Chenhao Wang Chao Li Zhehui Li Li Zeng Wei Wang QingE Wu |
| author_sort | Xiaoliang Qian |
| collection | DOAJ |
| description | Weakly supervised object detection (WSOD) has a great practical value in remote sensing image (RSI) interpretation because the instance-level annotations are not required. The multiple instance learning based methods are mainstream, and two problems should be addressed. First of all, the majority of methods usually detect discriminative parts rather than the whole object. Secondly, the quantity of easy instances is much greater than that of hard instances, which restricts the improvement of WSOD methods. To address the first problem, a multiscale image splitting based feature enhancement (MSFE) module is proposed. The MSFE module splits the input RSI in multiple scales, afterwards, the spatial attention maps (SAMs) are generated from the feature maps of each proposal corresponding to different splitting scales, and are used to calculate the maximum spatial attention map (MSAM). Each SAM is required to approach MSAM, which enforces the MSFE module to learn the feature maps which can highlight the whole object for each positive proposal. To address the second problem, an instance difficulty aware training (IDAT) strategy is proposed. The difficulty of each instance can be quantitatively measured, and is used as the weight of each instance in the training loss. Consequently, the hard instances will be focused in the training process. The ablation study demonstrates the validity of MSFE module and IDAT strategy. The comparisons with nine advanced methods on two RSI benchmarks further validate the overall effectiveness of our method. |
| first_indexed | 2024-03-12T14:02:34Z |
| format | Article |
| id | doaj.art-382294daf0504b12aa76df95d4953235 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-12T14:02:34Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-382294daf0504b12aa76df95d49532352023-08-21T23:00:18ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01167497750610.1109/JSTARS.2023.330441110214659Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing ImagesXiaoliang Qian0https://orcid.org/0000-0002-4328-6411Chenhao Wang1https://orcid.org/0009-0009-0612-9845Chao Li2https://orcid.org/0009-0009-5145-5156Zhehui Li3https://orcid.org/0009-0001-2130-5546Li Zeng4https://orcid.org/0009-0004-2867-0687Wei Wang5https://orcid.org/0000-0002-8770-3862QingE Wu6https://orcid.org/0000-0002-7746-8694College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaNetwork Technology Center, Henan Provincial Institute of Scientific and Technical Information, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, ChinaWeakly supervised object detection (WSOD) has a great practical value in remote sensing image (RSI) interpretation because the instance-level annotations are not required. The multiple instance learning based methods are mainstream, and two problems should be addressed. First of all, the majority of methods usually detect discriminative parts rather than the whole object. Secondly, the quantity of easy instances is much greater than that of hard instances, which restricts the improvement of WSOD methods. To address the first problem, a multiscale image splitting based feature enhancement (MSFE) module is proposed. The MSFE module splits the input RSI in multiple scales, afterwards, the spatial attention maps (SAMs) are generated from the feature maps of each proposal corresponding to different splitting scales, and are used to calculate the maximum spatial attention map (MSAM). Each SAM is required to approach MSAM, which enforces the MSFE module to learn the feature maps which can highlight the whole object for each positive proposal. To address the second problem, an instance difficulty aware training (IDAT) strategy is proposed. The difficulty of each instance can be quantitatively measured, and is used as the weight of each instance in the training loss. Consequently, the hard instances will be focused in the training process. The ablation study demonstrates the validity of MSFE module and IDAT strategy. The comparisons with nine advanced methods on two RSI benchmarks further validate the overall effectiveness of our method.https://ieeexplore.ieee.org/document/10214659/Instance difficulty aware training strategymultiscale image splitting based feature enhancement (MSFE)remote sensing image (RSI)weakly supervised object detection (WSOD) |
| spellingShingle | Xiaoliang Qian Chenhao Wang Chao Li Zhehui Li Li Zeng Wei Wang QingE Wu Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Instance difficulty aware training strategy multiscale image splitting based feature enhancement (MSFE) remote sensing image (RSI) weakly supervised object detection (WSOD) |
| title | Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images |
| title_full | Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images |
| title_fullStr | Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images |
| title_full_unstemmed | Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images |
| title_short | Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images |
| title_sort | multiscale image splitting based feature enhancement and instance difficulty aware training for weakly supervised object detection in remote sensing images |
| topic | Instance difficulty aware training strategy multiscale image splitting based feature enhancement (MSFE) remote sensing image (RSI) weakly supervised object detection (WSOD) |
| url | https://ieeexplore.ieee.org/document/10214659/ |
| work_keys_str_mv | AT xiaoliangqian multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages AT chenhaowang multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages AT chaoli multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages AT zhehuili multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages AT lizeng multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages AT weiwang multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages AT qingewu multiscaleimagesplittingbasedfeatureenhancementandinstancedifficultyawaretrainingforweaklysupervisedobjectdetectioninremotesensingimages |