Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data

Few-shot semantic segmentation (FSS) is committed to segmenting new classes with only a few labels. Generally, FSS assumes that base classes and novel classes belong to the same domain, which limits FSS’s application in a wide range of areas. In particular, since annotation is time-consuming, it is...

Full description

Bibliographic Details
Main Authors: Qingwei Sun, Jiangang Chao, Wanhong Lin, Zhenying Xu, Wei Chen, Ning He
Format: Article
Language:English
Published: MDPI AG 2023-10-01
Series:Remote Sensing
Subjects:
Online Access:https://www.mdpi.com/2072-4292/15/20/4937
_version_ 1797572409762512896
author Qingwei Sun
Jiangang Chao
Wanhong Lin
Zhenying Xu
Wei Chen
Ning He
author_facet Qingwei Sun
Jiangang Chao
Wanhong Lin
Zhenying Xu
Wei Chen
Ning He
author_sort Qingwei Sun
collection DOAJ
description Few-shot semantic segmentation (FSS) is committed to segmenting new classes with only a few labels. Generally, FSS assumes that base classes and novel classes belong to the same domain, which limits FSS’s application in a wide range of areas. In particular, since annotation is time-consuming, it is not cost-effective to process remote sensing images using FSS. To address this issue, we designed a feature transformation network (FTNet) for learning to few-shot segment remote sensing images from irrelevant data (FSS-RSI). The main idea is to train networks on irrelevant, already labeled data but inference on remote sensing images. In other words, the training and testing data neither belong to the same domain nor category. The FTNet contains two main modules: a feature transformation module (FTM) and a hierarchical transformer module (HTM). Among them, the FTM transforms features into a domain-agnostic high-level anchor, and the HTM hierarchically enhances matching between support and query features. Moreover, to promote the development of FSS-RSI, we established a new benchmark, which other researchers may use. Our experiments demonstrate that our model outperforms the cutting-edge few-shot semantic segmentation method by 25.39% and 21.31% in the one-shot and five-shot settings, respectively.
first_indexed 2024-03-10T20:55:52Z
format Article
id doaj.art-9fdb2996806b4cac9e165d724e418e43
institution Directory Open Access Journal
issn 2072-4292
language English
last_indexed 2024-03-10T20:55:52Z
publishDate 2023-10-01
publisher MDPI AG
record_format Article
series Remote Sensing
spelling doaj.art-9fdb2996806b4cac9e165d724e418e432023-11-19T17:58:30ZengMDPI AGRemote Sensing2072-42922023-10-011520493710.3390/rs15204937Learn to Few-Shot Segment Remote Sensing Images from Irrelevant DataQingwei Sun0Jiangang Chao1Wanhong Lin2Zhenying Xu3Wei Chen4Ning He5Department of Aerospace Science and Technology, Space Engineering University, Beijing 101416, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaChina Astronaut Research and Training Center, Beijing 100094, ChinaFew-shot semantic segmentation (FSS) is committed to segmenting new classes with only a few labels. Generally, FSS assumes that base classes and novel classes belong to the same domain, which limits FSS’s application in a wide range of areas. In particular, since annotation is time-consuming, it is not cost-effective to process remote sensing images using FSS. To address this issue, we designed a feature transformation network (FTNet) for learning to few-shot segment remote sensing images from irrelevant data (FSS-RSI). The main idea is to train networks on irrelevant, already labeled data but inference on remote sensing images. In other words, the training and testing data neither belong to the same domain nor category. The FTNet contains two main modules: a feature transformation module (FTM) and a hierarchical transformer module (HTM). Among them, the FTM transforms features into a domain-agnostic high-level anchor, and the HTM hierarchically enhances matching between support and query features. Moreover, to promote the development of FSS-RSI, we established a new benchmark, which other researchers may use. Our experiments demonstrate that our model outperforms the cutting-edge few-shot semantic segmentation method by 25.39% and 21.31% in the one-shot and five-shot settings, respectively.https://www.mdpi.com/2072-4292/15/20/4937meta-learningcross-domain segmentationfew-shot semantic segmentationtransformer
spellingShingle Qingwei Sun
Jiangang Chao
Wanhong Lin
Zhenying Xu
Wei Chen
Ning He
Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
Remote Sensing
meta-learning
cross-domain segmentation
few-shot semantic segmentation
transformer
title Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
title_full Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
title_fullStr Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
title_full_unstemmed Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
title_short Learn to Few-Shot Segment Remote Sensing Images from Irrelevant Data
title_sort learn to few shot segment remote sensing images from irrelevant data
topic meta-learning
cross-domain segmentation
few-shot semantic segmentation
transformer
url https://www.mdpi.com/2072-4292/15/20/4937
work_keys_str_mv AT qingweisun learntofewshotsegmentremotesensingimagesfromirrelevantdata
AT jiangangchao learntofewshotsegmentremotesensingimagesfromirrelevantdata
AT wanhonglin learntofewshotsegmentremotesensingimagesfromirrelevantdata
AT zhenyingxu learntofewshotsegmentremotesensingimagesfromirrelevantdata
AT weichen learntofewshotsegmentremotesensingimagesfromirrelevantdata
AT ninghe learntofewshotsegmentremotesensingimagesfromirrelevantdata