Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
Few-shot semantic segmentation (FSS) is committed to segmenting new classes with only a few labels. Generally, FSS assumes that base classes and novel classes belong to the same domain, which limits FSS’s application in a wide range of areas. In particular, since annotation is time-consuming, it is...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2023-10-01
|
Series: | Remote Sensing |
Subjects: | |
Online Access: | https://www.mdpi.com/2072-4292/15/20/4937 |
_version_ | 1797572409762512896 |
---|---|
author | Qingwei Sun Jiangang Chao Wanhong Lin Zhenying Xu Wei Chen Ning He |
author_facet | Qingwei Sun Jiangang Chao Wanhong Lin Zhenying Xu Wei Chen Ning He |
author_sort | Qingwei Sun |
collection | DOAJ |
description | Few-shot semantic segmentation (FSS) is committed to segmenting new classes with only a few labels. Generally, FSS assumes that base classes and novel classes belong to the same domain, which limits FSS’s application in a wide range of areas. In particular, since annotation is time-consuming, it is not cost-effective to process remote sensing images using FSS. To address this issue, we designed a feature transformation network (FTNet) for learning to few-shot segment remote sensing images from irrelevant data (FSS-RSI). The main idea is to train networks on irrelevant, already labeled data but inference on remote sensing images. In other words, the training and testing data neither belong to the same domain nor category. The FTNet contains two main modules: a feature transformation module (FTM) and a hierarchical transformer module (HTM). Among them, the FTM transforms features into a domain-agnostic high-level anchor, and the HTM hierarchically enhances matching between support and query features. Moreover, to promote the development of FSS-RSI, we established a new benchmark, which other researchers may use. Our experiments demonstrate that our model outperforms the cutting-edge few-shot semantic segmentation method by 25.39% and 21.31% in the one-shot and five-shot settings, respectively. |
first_indexed | 2024-03-10T20:55:52Z |
format | Article |
id | doaj.art-9fdb2996806b4cac9e165d724e418e43 |
institution | Directory Open Access Journal |
issn | 2072-4292 |
language | English |
last_indexed | 2024-03-10T20:55:52Z |
publishDate | 2023-10-01 |
publisher | MDPI AG |
record_format | Article |
series | Remote Sensing |
spelling | doaj.art-9fdb2996806b4cac9e165d724e418e432023-11-19T17:58:30ZengMDPI AGRemote Sensing2072-42922023-10-011520493710.3390/rs15204937Learn to Few-Shot Segment Remote Sensing Images from Irrelevant DataQingwei Sun0Jiangang Chao1Wanhong Lin2Zhenying Xu3Wei Chen4Ning He5Department of Aerospace Science and Technology, Space Engineering University, Beijing 101416, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaFew-shot semantic segmentation (FSS) is committed to segmenting new classes with only a few labels. Generally, FSS assumes that base classes and novel classes belong to the same domain, which limits FSS’s application in a wide range of areas. In particular, since annotation is time-consuming, it is not cost-effective to process remote sensing images using FSS. To address this issue, we designed a feature transformation network (FTNet) for learning to few-shot segment remote sensing images from irrelevant data (FSS-RSI). The main idea is to train networks on irrelevant, already labeled data but inference on remote sensing images. In other words, the training and testing data neither belong to the same domain nor category. The FTNet contains two main modules: a feature transformation module (FTM) and a hierarchical transformer module (HTM). Among them, the FTM transforms features into a domain-agnostic high-level anchor, and the HTM hierarchically enhances matching between support and query features. Moreover, to promote the development of FSS-RSI, we established a new benchmark, which other researchers may use. Our experiments demonstrate that our model outperforms the cutting-edge few-shot semantic segmentation method by 25.39% and 21.31% in the one-shot and five-shot settings, respectively.https://www.mdpi.com/2072-4292/15/20/4937meta-learningcross-domain segmentationfew-shot semantic segmentationtransformer |
spellingShingle | Qingwei Sun Jiangang Chao Wanhong Lin Zhenying Xu Wei Chen Ning He Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data Remote Sensing meta-learning cross-domain segmentation few-shot semantic segmentation transformer |
title | Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data |
title_full | Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data |
title_fullStr | Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data |
title_full_unstemmed | Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data |
title_short | Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data |
title_sort | learn to few shot segment remote sensing images from irrelevant data |
topic | meta-learning cross-domain segmentation few-shot semantic segmentation transformer |
url | https://www.mdpi.com/2072-4292/15/20/4937 |
work_keys_str_mv | AT qingweisun learntofewshotsegmentremotesensingimagesfromirrelevantdata AT jiangangchao learntofewshotsegmentremotesensingimagesfromirrelevantdata AT wanhonglin learntofewshotsegmentremotesensingimagesfromirrelevantdata AT zhenyingxu learntofewshotsegmentremotesensingimagesfromirrelevantdata AT weichen learntofewshotsegmentremotesensingimagesfromirrelevantdata AT ninghe learntofewshotsegmentremotesensingimagesfromirrelevantdata |