Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models
Semantic image segmentation is a core task for autonomous driving, which is performed by deep models. Since training these models draws to a curse of human-based image labeling, the use of synthetic images with automatically generated labels together with unlabeled real-world images is a promising a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/2/621 |
| _version_ | 1827622184548302848 |
|---|---|
| author | Jose L. Gómez Gabriel Villalonga Antonio M. López |
| author_facet | Jose L. Gómez Gabriel Villalonga Antonio M. López |
| author_sort | Jose L. Gómez |
| collection | DOAJ |
| description | Semantic image segmentation is a core task for autonomous driving, which is performed by deep models. Since training these models draws to a curse of human-based image labeling, the use of synthetic images with automatically generated labels together with unlabeled real-world images is a promising alternative. This implies addressing an unsupervised domain adaptation (UDA) problem. In this paper, we propose a new <i>co-training</i> procedure for synth-to-real UDA of semantic segmentation models. It performs iterations where the (unlabeled) real-world training images are labeled by intermediate deep models trained with both the (labeled) synthetic images and the real-world ones labeled in previous iterations. More specifically, a self-training stage provides two domain-adapted models and a model collaboration loop allows the mutual improvement of these two models. The final semantic segmentation labels (pseudo-labels) for the real-world images are provided by these two models. The overall procedure treats the deep models as black boxes and drives their collaboration at the level of pseudo-labeled target images, i.e., neither modifying loss functions is required, nor explicit feature alignment. We test our proposal on standard synthetic and real-world datasets for onboard semantic segmentation. Our procedure shows improvements ranging from approximately 13 to 31 mIoU points over baselines. |
| first_indexed | 2024-03-09T11:17:58Z |
| format | Article |
| id | doaj.art-3139cd74e8f84e0799ca2dfe973988ae |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T11:17:58Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-3139cd74e8f84e0799ca2dfe973988ae2023-12-01T00:24:46ZengMDPI AGSensors1424-82202023-01-0123262110.3390/s23020621Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation ModelsJose L. Gómez0Gabriel Villalonga1Antonio M. López2Computer Vision Center (CVC), Universitat Autònoma de Barcelona (UAB), 08193 Bellaterra, SpainComputer Vision Center (CVC), Universitat Autònoma de Barcelona (UAB), 08193 Bellaterra, SpainComputer Vision Center (CVC), Universitat Autònoma de Barcelona (UAB), 08193 Bellaterra, SpainSemantic image segmentation is a core task for autonomous driving, which is performed by deep models. Since training these models draws to a curse of human-based image labeling, the use of synthetic images with automatically generated labels together with unlabeled real-world images is a promising alternative. This implies addressing an unsupervised domain adaptation (UDA) problem. In this paper, we propose a new <i>co-training</i> procedure for synth-to-real UDA of semantic segmentation models. It performs iterations where the (unlabeled) real-world training images are labeled by intermediate deep models trained with both the (labeled) synthetic images and the real-world ones labeled in previous iterations. More specifically, a self-training stage provides two domain-adapted models and a model collaboration loop allows the mutual improvement of these two models. The final semantic segmentation labels (pseudo-labels) for the real-world images are provided by these two models. The overall procedure treats the deep models as black boxes and drives their collaboration at the level of pseudo-labeled target images, i.e., neither modifying loss functions is required, nor explicit feature alignment. We test our proposal on standard synthetic and real-world datasets for onboard semantic segmentation. Our procedure shows improvements ranging from approximately 13 to 31 mIoU points over baselines.https://www.mdpi.com/1424-8220/23/2/621domain adaptationsemi-supervised learningsemantic segmentationautonomous driving |
| spellingShingle | Jose L. Gómez Gabriel Villalonga Antonio M. López Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models Sensors domain adaptation semi-supervised learning semantic segmentation autonomous driving |
| title | Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models |
| title_full | Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models |
| title_fullStr | Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models |
| title_full_unstemmed | Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models |
| title_short | Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models |
| title_sort | co training for unsupervised domain adaptation of semantic segmentation models |
| topic | domain adaptation semi-supervised learning semantic segmentation autonomous driving |
| url | https://www.mdpi.com/1424-8220/23/2/621 |
| work_keys_str_mv | AT joselgomez cotrainingforunsuperviseddomainadaptationofsemanticsegmentationmodels AT gabrielvillalonga cotrainingforunsuperviseddomainadaptationofsemanticsegmentationmodels AT antoniomlopez cotrainingforunsuperviseddomainadaptationofsemanticsegmentationmodels |