Dual Prototype Learning for Few Shot Semantic Segmentation
Few-shot segmentation (FSS) is a challenging task because the same class of targets in the support and query images may have different scales, textures and background information. Prototype learning (PL) is a current mainstream FSS method, which characterizes the interaction between the prototype ve...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10382511/ |
| _version_ | 1827382785003749376 |
|---|---|
| author | Wenxuan Li Shaobo Chen Chengyi Xiong |
| author_facet | Wenxuan Li Shaobo Chen Chengyi Xiong |
| author_sort | Wenxuan Li |
| collection | DOAJ |
| description | Few-shot segmentation (FSS) is a challenging task because the same class of targets in the support and query images may have different scales, textures and background information. Prototype learning (PL) is a current mainstream FSS method, which characterizes the interaction between the prototype vector and query feature. However, the prototype vector commonly based on global average pooling only contains first-order feature information, which is vulnerable to varying appearance of similar target and the diversity of background. Moreover, the auxiliary information of the query image is not fully explored in previous prototype learning methods. In this paper, we propose a dual prototype learning (DPL) based on a second-order prototype (SOP) and self-support first-order prototype with a constraint mechanism (SSFPC) to improve the FSS performance. The SOP can capture higher-order statistical information by averaging the covariance matrix of the feature map. The similarity between the first-order support prototype and the first-order self-support query prototype is introduced to boost the adaptability of the first-order prototype to the query image. The remarkable performance gains on the benchmarks (PASCAL-<inline-formula> <tex-math notation="LaTeX">${5^{i}}$ </tex-math></inline-formula> and COCO-<inline-formula> <tex-math notation="LaTeX">${20^{i}}$ </tex-math></inline-formula>) manifest the effectiveness of our method. Our source code will be available at <uri>https://github.com/13ww/DPL.git</uri>. |
| first_indexed | 2024-03-08T14:27:28Z |
| format | Article |
| id | doaj.art-c0f9046f178e4df9b90aff2dd87c1c01 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T14:27:28Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c0f9046f178e4df9b90aff2dd87c1c012024-01-13T00:02:26ZengIEEEIEEE Access2169-35362024-01-01126356636410.1109/ACCESS.2024.335074710382511Dual Prototype Learning for Few Shot Semantic SegmentationWenxuan Li0https://orcid.org/0009-0008-7675-8711Shaobo Chen1https://orcid.org/0009-0005-5710-7284Chengyi Xiong2https://orcid.org/0000-0002-0831-3437School of Electronic and Information Engineering, South-Central Minzu University, Wuhan, ChinaSchool of Electronic and Information Engineering, South-Central Minzu University, Wuhan, ChinaSchool of Electronic and Information Engineering, South-Central Minzu University, Wuhan, ChinaFew-shot segmentation (FSS) is a challenging task because the same class of targets in the support and query images may have different scales, textures and background information. Prototype learning (PL) is a current mainstream FSS method, which characterizes the interaction between the prototype vector and query feature. However, the prototype vector commonly based on global average pooling only contains first-order feature information, which is vulnerable to varying appearance of similar target and the diversity of background. Moreover, the auxiliary information of the query image is not fully explored in previous prototype learning methods. In this paper, we propose a dual prototype learning (DPL) based on a second-order prototype (SOP) and self-support first-order prototype with a constraint mechanism (SSFPC) to improve the FSS performance. The SOP can capture higher-order statistical information by averaging the covariance matrix of the feature map. The similarity between the first-order support prototype and the first-order self-support query prototype is introduced to boost the adaptability of the first-order prototype to the query image. The remarkable performance gains on the benchmarks (PASCAL-<inline-formula> <tex-math notation="LaTeX">${5^{i}}$ </tex-math></inline-formula> and COCO-<inline-formula> <tex-math notation="LaTeX">${20^{i}}$ </tex-math></inline-formula>) manifest the effectiveness of our method. Our source code will be available at <uri>https://github.com/13ww/DPL.git</uri>.https://ieeexplore.ieee.org/document/10382511/Few-shot learningsemantic segmentationfirst-order prototypesecond-order prototype |
| spellingShingle | Wenxuan Li Shaobo Chen Chengyi Xiong Dual Prototype Learning for Few Shot Semantic Segmentation IEEE Access Few-shot learning semantic segmentation first-order prototype second-order prototype |
| title | Dual Prototype Learning for Few Shot Semantic Segmentation |
| title_full | Dual Prototype Learning for Few Shot Semantic Segmentation |
| title_fullStr | Dual Prototype Learning for Few Shot Semantic Segmentation |
| title_full_unstemmed | Dual Prototype Learning for Few Shot Semantic Segmentation |
| title_short | Dual Prototype Learning for Few Shot Semantic Segmentation |
| title_sort | dual prototype learning for few shot semantic segmentation |
| topic | Few-shot learning semantic segmentation first-order prototype second-order prototype |
| url | https://ieeexplore.ieee.org/document/10382511/ |
| work_keys_str_mv | AT wenxuanli dualprototypelearningforfewshotsemanticsegmentation AT shaobochen dualprototypelearningforfewshotsemanticsegmentation AT chengyixiong dualprototypelearningforfewshotsemanticsegmentation |