SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images
Semantic segmentation based on optical images can provide comprehensive scene information for intelligent vehicle systems, thus aiding in scene perception and decision making. However, under adverse weather conditions (such as fog), the performance of methods can be compromised due to incomplete obs...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/24/5704 |
| _version_ | 1797379485750788096 |
|---|---|
| author | Ziquan Wang Yongsheng Zhang Zhenchao Zhang Zhipeng Jiang Ying Yu Li Li Lei Zhang |
| author_facet | Ziquan Wang Yongsheng Zhang Zhenchao Zhang Zhipeng Jiang Ying Yu Li Li Lei Zhang |
| author_sort | Ziquan Wang |
| collection | DOAJ |
| description | Semantic segmentation based on optical images can provide comprehensive scene information for intelligent vehicle systems, thus aiding in scene perception and decision making. However, under adverse weather conditions (such as fog), the performance of methods can be compromised due to incomplete observations. Considering the success of domain adaptation in recent years, we believe it is reasonable to transfer knowledge from clear and existing annotated datasets to images with fog. Technically, we follow the main workflow of the previous SDAT-Former method, which incorporates fog and style-factor knowledge into the teacher segmentor to generate better pseudo-labels for guiding the student segmentor, but we identify and address some issues, achieving significant improvements. Firstly, we introduce a consistency loss for learning from multiple source data to better converge the performance of each component. Secondly, we apply positional encoding to the features of fog-invariant adversarial learning, strengthening the model’s ability to handle the details of foggy entities. Furthermore, to address the complexity and noise in the original version, we integrate a simple but effective masked learning technique into a unified, end-to-end training process. Finally, we regularize the knowledge transfer in the original method through re-weighting. We tested our SDAT-Former++ on mainstream benchmarks for semantic segmentation in foggy scenes, demonstrating improvements of 3.3%, 4.8%, and 1.1% (as measured by the mIoU) on the ACDC, Foggy Zurich, and Foggy Driving datasets, respectively, compared to the original version. |
| first_indexed | 2024-03-08T20:23:58Z |
| format | Article |
| id | doaj.art-4c5f703ab847475887184b1c524aa89a |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-08T20:23:58Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-4c5f703ab847475887184b1c524aa89a2023-12-22T14:39:05ZengMDPI AGRemote Sensing2072-42922023-12-011524570410.3390/rs15245704SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing ImagesZiquan Wang0Yongsheng Zhang1Zhenchao Zhang2Zhipeng Jiang3Ying Yu4Li Li5Lei Zhang6School of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSchool of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSchool of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSchool of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSchool of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSchool of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSchool of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaSemantic segmentation based on optical images can provide comprehensive scene information for intelligent vehicle systems, thus aiding in scene perception and decision making. However, under adverse weather conditions (such as fog), the performance of methods can be compromised due to incomplete observations. Considering the success of domain adaptation in recent years, we believe it is reasonable to transfer knowledge from clear and existing annotated datasets to images with fog. Technically, we follow the main workflow of the previous SDAT-Former method, which incorporates fog and style-factor knowledge into the teacher segmentor to generate better pseudo-labels for guiding the student segmentor, but we identify and address some issues, achieving significant improvements. Firstly, we introduce a consistency loss for learning from multiple source data to better converge the performance of each component. Secondly, we apply positional encoding to the features of fog-invariant adversarial learning, strengthening the model’s ability to handle the details of foggy entities. Furthermore, to address the complexity and noise in the original version, we integrate a simple but effective masked learning technique into a unified, end-to-end training process. Finally, we regularize the knowledge transfer in the original method through re-weighting. We tested our SDAT-Former++ on mainstream benchmarks for semantic segmentation in foggy scenes, demonstrating improvements of 3.3%, 4.8%, and 1.1% (as measured by the mIoU) on the ACDC, Foggy Zurich, and Foggy Driving datasets, respectively, compared to the original version.https://www.mdpi.com/2072-4292/15/24/5704semantic segmentation in foggy scenesunsupervised domain adaptationUDAself-training |
| spellingShingle | Ziquan Wang Yongsheng Zhang Zhenchao Zhang Zhipeng Jiang Ying Yu Li Li Lei Zhang SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images Remote Sensing semantic segmentation in foggy scenes unsupervised domain adaptation UDA self-training |
| title | SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images |
| title_full | SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images |
| title_fullStr | SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images |
| title_full_unstemmed | SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images |
| title_short | SDAT-Former++: A Foggy Scene Semantic Segmentation Method with Stronger Domain Adaption Teacher for Remote Sensing Images |
| title_sort | sdat former a foggy scene semantic segmentation method with stronger domain adaption teacher for remote sensing images |
| topic | semantic segmentation in foggy scenes unsupervised domain adaptation UDA self-training |
| url | https://www.mdpi.com/2072-4292/15/24/5704 |
| work_keys_str_mv | AT ziquanwang sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages AT yongshengzhang sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages AT zhenchaozhang sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages AT zhipengjiang sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages AT yingyu sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages AT lili sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages AT leizhang sdatformerafoggyscenesemanticsegmentationmethodwithstrongerdomainadaptionteacherforremotesensingimages |