MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries
Weakly supervised semantic segmentation (WSSS) methods based on image-level labels can relieve the tedious pixel-level annotation burden, and these methods are mainly based on class activation maps (CAMs). However, it is challenging to generate high-quality CAMs for high-resolution remotely sensed i...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9665252/ |
| _version_ | 1818985471704825856 |
|---|---|
| author | Xin Yan Li Shen Jicheng Wang Xu Deng Zhilin Li |
| author_facet | Xin Yan Li Shen Jicheng Wang Xu Deng Zhilin Li |
| author_sort | Xin Yan |
| collection | DOAJ |
| description | Weakly supervised semantic segmentation (WSSS) methods based on image-level labels can relieve the tedious pixel-level annotation burden, and these methods are mainly based on class activation maps (CAMs). However, it is challenging to generate high-quality CAMs for high-resolution remotely sensed imagery (HRSI). In this article, we propose a WSSS method for building extraction from HRSI using image-level labels. The proposed method, termed as the MSG-SR-Net, integrates two novel modules, i.e., multiscale generation (MSG) and superpixel refinement (SR), to obtain high-quality CAMs so as to provide reliable pixel-level training samples for subsequent semantic segmentation steps. The MSG module is proposed to use global semantic information to guide the learning of multiple features across different levels, and then, respectively, to utilize multilevel features for generating multiscale CAMs. This component can effectively suppress the interference of the class-irrelevant noise and strengthen the use of profitable information in multilevel features. The SR module is designed to take advantage of superpixels to improve multiscale CAMs in target integrity and details preserving. Extensive experiments on two public building datasets demonstrated that the proposed modules made the MSG-SR-Net obtain more integral and accurate CAMs for building extraction. Moreover, experimental results also showed the proposed method achieved excellent performance with over 67% in F1-score, and outperformed other weakly supervised methods in effectiveness and generalization ability. |
| first_indexed | 2024-12-20T18:35:26Z |
| format | Article |
| id | doaj.art-38630e395cbd42ae898df84f154fcf32 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-20T18:35:26Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-38630e395cbd42ae898df84f154fcf322022-12-21T19:29:56ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352022-01-01151012102310.1109/JSTARS.2021.31374509665252MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed ImageriesXin Yan0Li Shen1https://orcid.org/0000-0003-4638-3329Jicheng Wang2Xu Deng3Zhilin Li4State-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu, ChinaState-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu, ChinaKey Laboratory of Ministry of Education on Land Resources Evaluation and Monitoring in Southwest China, Sichuan Normal University, Chengdu, ChinaState-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu, ChinaState-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu, ChinaWeakly supervised semantic segmentation (WSSS) methods based on image-level labels can relieve the tedious pixel-level annotation burden, and these methods are mainly based on class activation maps (CAMs). However, it is challenging to generate high-quality CAMs for high-resolution remotely sensed imagery (HRSI). In this article, we propose a WSSS method for building extraction from HRSI using image-level labels. The proposed method, termed as the MSG-SR-Net, integrates two novel modules, i.e., multiscale generation (MSG) and superpixel refinement (SR), to obtain high-quality CAMs so as to provide reliable pixel-level training samples for subsequent semantic segmentation steps. The MSG module is proposed to use global semantic information to guide the learning of multiple features across different levels, and then, respectively, to utilize multilevel features for generating multiscale CAMs. This component can effectively suppress the interference of the class-irrelevant noise and strengthen the use of profitable information in multilevel features. The SR module is designed to take advantage of superpixels to improve multiscale CAMs in target integrity and details preserving. Extensive experiments on two public building datasets demonstrated that the proposed modules made the MSG-SR-Net obtain more integral and accurate CAMs for building extraction. Moreover, experimental results also showed the proposed method achieved excellent performance with over 67% in F1-score, and outperformed other weakly supervised methods in effectiveness and generalization ability.https://ieeexplore.ieee.org/document/9665252/Building extractionclass activation maphigh-resolution remotely sensed imagerysuperpixel refinementweakly supervised deep learning |
| spellingShingle | Xin Yan Li Shen Jicheng Wang Xu Deng Zhilin Li MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Building extraction class activation map high-resolution remotely sensed imagery superpixel refinement weakly supervised deep learning |
| title | MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries |
| title_full | MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries |
| title_fullStr | MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries |
| title_full_unstemmed | MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries |
| title_short | MSG-SR-Net: A Weakly Supervised Network Integrating Multiscale Generation and Superpixel Refinement for Building Extraction From High-Resolution Remotely Sensed Imageries |
| title_sort | msg sr net a weakly supervised network integrating multiscale generation and superpixel refinement for building extraction from high resolution remotely sensed imageries |
| topic | Building extraction class activation map high-resolution remotely sensed imagery superpixel refinement weakly supervised deep learning |
| url | https://ieeexplore.ieee.org/document/9665252/ |
| work_keys_str_mv | AT xinyan msgsrnetaweaklysupervisednetworkintegratingmultiscalegenerationandsuperpixelrefinementforbuildingextractionfromhighresolutionremotelysensedimageries AT lishen msgsrnetaweaklysupervisednetworkintegratingmultiscalegenerationandsuperpixelrefinementforbuildingextractionfromhighresolutionremotelysensedimageries AT jichengwang msgsrnetaweaklysupervisednetworkintegratingmultiscalegenerationandsuperpixelrefinementforbuildingextractionfromhighresolutionremotelysensedimageries AT xudeng msgsrnetaweaklysupervisednetworkintegratingmultiscalegenerationandsuperpixelrefinementforbuildingextractionfromhighresolutionremotelysensedimageries AT zhilinli msgsrnetaweaklysupervisednetworkintegratingmultiscalegenerationandsuperpixelrefinementforbuildingextractionfromhighresolutionremotelysensedimageries |