CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning
Recent progress in semantic segmentation is driven by deep Convolutional Neural Networks and large-scale labeled image datasets. However, data labeling for pixel-wise segmentation is tedious and costly. Moreover, a trained model can only make predictions within a set of pre-defined classes. In this...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/144391 |
| _version_ | 1811679357146497024 |
|---|---|
| author | Zhang, Chi Lin, Guosheng Liu, Fayao Yao, Rui Shen, Chunhua |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Zhang, Chi Lin, Guosheng Liu, Fayao Yao, Rui Shen, Chunhua |
| author_sort | Zhang, Chi |
| collection | NTU |
| description | Recent progress in semantic segmentation is driven by deep Convolutional Neural Networks and large-scale labeled image datasets. However, data labeling for pixel-wise segmentation is tedious and costly. Moreover, a trained model can only make predictions within a set of pre-defined classes. In this paper, we present CANet, a class-agnostic segmentation network that performs few-shot segmentation on new classes with only a few annotated images available. Our network consists of a two-branch dense comparison module which performs multi-level feature comparison between the support image and the query image, and an iterative optimization module which iteratively refines the predicted results. Furthermore, we introduce an attention mechanism to effectively fuse information from multiple support examples under the setting of k-shot learning. Experiments on PASCAL VOC 2012 show that our method achieves a mean Intersection-over-Union score of 55.4% for 1-shot segmentation and 57.1% for 5-shot segmentation, outperforming state-of-the-art methods by a large margin of 14.6% and 13.2%, respectively. |
| first_indexed | 2024-10-01T03:07:52Z |
| format | Conference Paper |
| id | ntu-10356/144391 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:07:52Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1443912020-11-03T02:29:28Z CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning Zhang, Chi Lin, Guosheng Liu, Fayao Yao, Rui Shen, Chunhua School of Computer Science and Engineering 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Engineering::Computer science and engineering Retrieval Segmentation Recent progress in semantic segmentation is driven by deep Convolutional Neural Networks and large-scale labeled image datasets. However, data labeling for pixel-wise segmentation is tedious and costly. Moreover, a trained model can only make predictions within a set of pre-defined classes. In this paper, we present CANet, a class-agnostic segmentation network that performs few-shot segmentation on new classes with only a few annotated images available. Our network consists of a two-branch dense comparison module which performs multi-level feature comparison between the support image and the query image, and an iterative optimization module which iteratively refines the predicted results. Furthermore, we introduce an attention mechanism to effectively fuse information from multiple support examples under the setting of k-shot learning. Experiments on PASCAL VOC 2012 show that our method achieves a mean Intersection-over-Union score of 55.4% for 1-shot segmentation and 57.1% for 5-shot segmentation, outperforming state-of-the-art methods by a large margin of 14.6% and 13.2%, respectively. AI Singapore Ministry of Education (MOE) Accepted version G. Lin’s participation was partly supported by the National Research Foundation Singapore under its AI Singapore Programme [AISG-RP-2018-003] and a MOE Tier-1 research grant [RG126/17 (S)]. R. Yao’s participation was supported by the National Natural Scientific Foundation of China (NSFC) under Grant No. 61772530. We would like to thank NVIDIA for GPU donation. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. 2020-11-03T02:29:27Z 2020-11-03T02:29:27Z 2019 Conference Paper Zhang, C., Lin, G., Liu, F., Yao, R., & Shen, C. (2019). CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/CVPR.2019.00536 https://hdl.handle.net/10356/144391 10.1109/CVPR.2019.00536 en AISG-RP-2018-003 RG126/17 (S) © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work is available at: https://doi.org/10.1109/CVPR.2019.00536 application/pdf |
| spellingShingle | Engineering::Computer science and engineering Retrieval Segmentation Zhang, Chi Lin, Guosheng Liu, Fayao Yao, Rui Shen, Chunhua CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning |
| title | CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning |
| title_full | CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning |
| title_fullStr | CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning |
| title_full_unstemmed | CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning |
| title_short | CANet : class-agnostic segmentation networks with iterative refinement and attentive few-shot learning |
| title_sort | canet class agnostic segmentation networks with iterative refinement and attentive few shot learning |
| topic | Engineering::Computer science and engineering Retrieval Segmentation |
| url | https://hdl.handle.net/10356/144391 |
| work_keys_str_mv | AT zhangchi canetclassagnosticsegmentationnetworkswithiterativerefinementandattentivefewshotlearning AT linguosheng canetclassagnosticsegmentationnetworkswithiterativerefinementandattentivefewshotlearning AT liufayao canetclassagnosticsegmentationnetworkswithiterativerefinementandattentivefewshotlearning AT yaorui canetclassagnosticsegmentationnetworkswithiterativerefinementandattentivefewshotlearning AT shenchunhua canetclassagnosticsegmentationnetworkswithiterativerefinementandattentivefewshotlearning |