CRNet : cross-reference networks for few-shot segmentation
Over the past few years, state-of-the-art image segmentation algorithms are based on deep convolutional neural networks. To render a deep network with the ability to understand a concept, humans need to collect a large amount of pixel-level annotated data to train the models, which is time-consuming...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/144247 |
| _version_ | 1826114184838905856 |
|---|---|
| author | Liu, Weide Zhang, Chi Lin, Guosheng Liu, Fayao |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Liu, Weide Zhang, Chi Lin, Guosheng Liu, Fayao |
| author_sort | Liu, Weide |
| collection | NTU |
| description | Over the past few years, state-of-the-art image segmentation algorithms are based on deep convolutional neural networks. To render a deep network with the ability to understand a concept, humans need to collect a large amount of pixel-level annotated data to train the models, which is time-consuming and tedious. Recently, few-shot segmentation is proposed to solve this problem. Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. In this paper, we propose a cross-reference network (CRNet) for few-shot segmentation. Unlike previous works which only predict the mask in the query image, our proposed model concurrently make predictions for both the support image and the query image. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images, thus helping the few-shot segmentation task. We also develop a mask refinement module to recurrently refine the prediction of the foreground regions. For the k-shot learning, we propose to finetune parts of networks to take advantage of multiple labeled support images. Experiments on the PASCAL VOC 2012 dataset show that our network
achieves state-of-the-art performance. |
| first_indexed | 2024-10-01T03:35:21Z |
| format | Conference Paper |
| id | ntu-10356/144247 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:35:21Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1442472020-10-23T02:33:59Z CRNet : cross-reference networks for few-shot segmentation Liu, Weide Zhang, Chi Lin, Guosheng Liu, Fayao School of Computer Science and Engineering 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Engineering::Computer science and engineering Image Segmentation Predictive Models Over the past few years, state-of-the-art image segmentation algorithms are based on deep convolutional neural networks. To render a deep network with the ability to understand a concept, humans need to collect a large amount of pixel-level annotated data to train the models, which is time-consuming and tedious. Recently, few-shot segmentation is proposed to solve this problem. Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. In this paper, we propose a cross-reference network (CRNet) for few-shot segmentation. Unlike previous works which only predict the mask in the query image, our proposed model concurrently make predictions for both the support image and the query image. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images, thus helping the few-shot segmentation task. We also develop a mask refinement module to recurrently refine the prediction of the foreground regions. For the k-shot learning, we propose to finetune parts of networks to take advantage of multiple labeled support images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance. AI Singapore Ministry of Education (MOE) National Research Foundation (NRF) Accepted version This research is supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-RP-2018-003) and the MOE Tier-1 research grants: RG126/17 (S) and RG22/19 (S). This research is also partly supported by the Delta-NTU Corporate Lab with funding support from Delta Electronics Inc. and the National Research Foundation (NRF) Singapore. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore 2020-10-23T02:33:59Z 2020-10-23T02:33:59Z 2020 Conference Paper Liu, W., Zhang, C., Lin, G., & Liu, F. (2020). CRNet : cross-reference networks for few-shot segmentation. Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/CVPR42600.2020.00422 https://hdl.handle.net/10356/144247 10.1109/CVPR42600.2020.00422 en AISG-RP-2018-003 RG126/17 (S) RG22/19 (S) © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/CVPR42600.2020.00422 application/pdf |
| spellingShingle | Engineering::Computer science and engineering Image Segmentation Predictive Models Liu, Weide Zhang, Chi Lin, Guosheng Liu, Fayao CRNet : cross-reference networks for few-shot segmentation |
| title | CRNet : cross-reference networks for few-shot segmentation |
| title_full | CRNet : cross-reference networks for few-shot segmentation |
| title_fullStr | CRNet : cross-reference networks for few-shot segmentation |
| title_full_unstemmed | CRNet : cross-reference networks for few-shot segmentation |
| title_short | CRNet : cross-reference networks for few-shot segmentation |
| title_sort | crnet cross reference networks for few shot segmentation |
| topic | Engineering::Computer science and engineering Image Segmentation Predictive Models |
| url | https://hdl.handle.net/10356/144247 |
| work_keys_str_mv | AT liuweide crnetcrossreferencenetworksforfewshotsegmentation AT zhangchi crnetcrossreferencenetworksforfewshotsegmentation AT linguosheng crnetcrossreferencenetworksforfewshotsegmentation AT liufayao crnetcrossreferencenetworksforfewshotsegmentation |