Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer
With the development of deep learning, the performance of image semantic segmentation in remote sensing has been constantly improved. However, the performance usually degrades while testing on different datasets because of the domain gap. To achieve feasible performance, extensive pixel-wise annotat...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/19/4942 |
| _version_ | 1797477112819482624 |
|---|---|
| author | Weitao Li Hui Gao Yi Su Biffon Manyura Momanyi |
| author_facet | Weitao Li Hui Gao Yi Su Biffon Manyura Momanyi |
| author_sort | Weitao Li |
| collection | DOAJ |
| description | With the development of deep learning, the performance of image semantic segmentation in remote sensing has been constantly improved. However, the performance usually degrades while testing on different datasets because of the domain gap. To achieve feasible performance, extensive pixel-wise annotations are acquired in a new environment, which is time-consuming and labor-intensive. Therefore, unsupervised domain adaptation (UDA) has been proposed to alleviate the effort of labeling. However, most previous approaches are based on outdated network architectures that hinder the improvement of performance in UDA. Since the effects of recent architectures for UDA have been barely studied, we reveal the potential of Transformer in UDA for remote sensing with a self-training framework. Additionally, two training strategies have been proposed to enhance the performance of UDA: (1) Gradual Class Weights (GCW) to stabilize the model on the source domain by addressing the class-imbalance problem; (2) Local Dynamic Quality (LDQ) to improve the quality of the pseudo-labels via distinguishing the discrete and clustered pseudo-labels on the target domain. Overall, our proposed method improves the state-of-the-art performance by 8.23% mIoU on Potsdam→Vaihingen and 9.2% mIoU on Vaihingen→Potsdam and facilitates learning even for difficult classes such as clutter/background. |
| first_indexed | 2024-03-09T21:13:05Z |
| format | Article |
| id | doaj.art-84374946256446489a140c4a895ec5e8 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T21:13:05Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-84374946256446489a140c4a895ec5e82023-11-23T21:41:15ZengMDPI AGRemote Sensing2072-42922022-10-011419494210.3390/rs14194942Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with TransformerWeitao Li0Hui Gao1Yi Su2Biffon Manyura Momanyi3School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611700, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611700, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611700, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611700, ChinaWith the development of deep learning, the performance of image semantic segmentation in remote sensing has been constantly improved. However, the performance usually degrades while testing on different datasets because of the domain gap. To achieve feasible performance, extensive pixel-wise annotations are acquired in a new environment, which is time-consuming and labor-intensive. Therefore, unsupervised domain adaptation (UDA) has been proposed to alleviate the effort of labeling. However, most previous approaches are based on outdated network architectures that hinder the improvement of performance in UDA. Since the effects of recent architectures for UDA have been barely studied, we reveal the potential of Transformer in UDA for remote sensing with a self-training framework. Additionally, two training strategies have been proposed to enhance the performance of UDA: (1) Gradual Class Weights (GCW) to stabilize the model on the source domain by addressing the class-imbalance problem; (2) Local Dynamic Quality (LDQ) to improve the quality of the pseudo-labels via distinguishing the discrete and clustered pseudo-labels on the target domain. Overall, our proposed method improves the state-of-the-art performance by 8.23% mIoU on Potsdam→Vaihingen and 9.2% mIoU on Vaihingen→Potsdam and facilitates learning even for difficult classes such as clutter/background.https://www.mdpi.com/2072-4292/14/19/4942unsupervised domain adaptationsemantic segmentationremote sensing imagetransformerself-training |
| spellingShingle | Weitao Li Hui Gao Yi Su Biffon Manyura Momanyi Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer Remote Sensing unsupervised domain adaptation semantic segmentation remote sensing image transformer self-training |
| title | Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer |
| title_full | Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer |
| title_fullStr | Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer |
| title_full_unstemmed | Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer |
| title_short | Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer |
| title_sort | unsupervised domain adaptation for remote sensing semantic segmentation with transformer |
| topic | unsupervised domain adaptation semantic segmentation remote sensing image transformer self-training |
| url | https://www.mdpi.com/2072-4292/14/19/4942 |
| work_keys_str_mv | AT weitaoli unsuperviseddomainadaptationforremotesensingsemanticsegmentationwithtransformer AT huigao unsuperviseddomainadaptationforremotesensingsemanticsegmentationwithtransformer AT yisu unsuperviseddomainadaptationforremotesensingsemanticsegmentationwithtransformer AT biffonmanyuramomanyi unsuperviseddomainadaptationforremotesensingsemanticsegmentationwithtransformer |