Cross-image region mining with region prototypical network for weakly supervised segmentation
Weakly supervised image segmentation trained with image-level labels usually suffers from inaccurate coverage of object areas during the generation of the pseudo groundtruth. This is because the object activation maps are trained with the classification objective and lack the ability to generalize....
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/162959 |
| _version_ | 1826119783974699008 |
|---|---|
| author | Liu, Weide Kong, Xiangfei Hung, Tzu-Yi Lin, Guosheng |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Liu, Weide Kong, Xiangfei Hung, Tzu-Yi Lin, Guosheng |
| author_sort | Liu, Weide |
| collection | NTU |
| description | Weakly supervised image segmentation trained with image-level labels usually suffers from inaccurate coverage of object areas during the generation of the pseudo groundtruth. This is because the object activation maps are trained with the classification objective and lack the ability to generalize. To improve the generality of the objective activation maps, we propose a region prototypical network (RPNet) to explore the cross-image object diversity of the training set. Similar object parts across images are identified via region feature comparison. Object confidence is propagated between regions to discover new object areas while background regions are suppressed. Experiments show that the proposed method generates more complete and accurate pseudo object masks, while achieving state-of-the-art performance on PASCAL VOC 2012 and MS COCO. In addition, we investigate the robustness of the proposed method on reduced training sets. |
| first_indexed | 2024-10-01T05:05:46Z |
| format | Journal Article |
| id | ntu-10356/162959 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:05:46Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1629592022-11-14T01:28:52Z Cross-image region mining with region prototypical network for weakly supervised segmentation Liu, Weide Kong, Xiangfei Hung, Tzu-Yi Lin, Guosheng School of Computer Science and Engineering Institute for Infocomm Research (I2R) (A*STAR) Engineering::Computer science and engineering Cross-Image Segmentation Weakly supervised image segmentation trained with image-level labels usually suffers from inaccurate coverage of object areas during the generation of the pseudo groundtruth. This is because the object activation maps are trained with the classification objective and lack the ability to generalize. To improve the generality of the objective activation maps, we propose a region prototypical network (RPNet) to explore the cross-image object diversity of the training set. Similar object parts across images are identified via region feature comparison. Object confidence is propagated between regions to discover new object areas while background regions are suppressed. Experiments show that the proposed method generates more complete and accurate pseudo object masks, while achieving state-of-the-art performance on PASCAL VOC 2012 and MS COCO. In addition, we investigate the robustness of the proposed method on reduced training sets. Ministry of Education (MOE) National Research Foundation (NRF) This work is supported by the Delta-NTU Corporate Lab with funding support from Delta Electronics Inc. and the National Research Foundation (NRF) Singapore (SMA-RP10). This work is also partly supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-RP-2018-003) and the MOE Tier-1 research grants: RG28/18 (S), RG22/19 (S) and RG95/20, and the National Natural Science Foundation of China (No.61902077). 2022-11-14T01:28:51Z 2022-11-14T01:28:51Z 2021 Journal Article Liu, W., Kong, X., Hung, T. & Lin, G. (2021). Cross-image region mining with region prototypical network for weakly supervised segmentation. IEEE Transactions On Multimedia, 3139459-. https://dx.doi.org/10.1109/TMM.2021.3139459 1520-9210 https://hdl.handle.net/10356/162959 10.1109/TMM.2021.3139459 2-s2.0-85122562950 3139459 en SMA-RP10 AISG-RP-2018-003 RG28/18 (S) RG22/19 (S) RG95/20 IEEE Transactions on Multimedia © 2021 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Cross-Image Segmentation Liu, Weide Kong, Xiangfei Hung, Tzu-Yi Lin, Guosheng Cross-image region mining with region prototypical network for weakly supervised segmentation |
| title | Cross-image region mining with region prototypical network for weakly supervised segmentation |
| title_full | Cross-image region mining with region prototypical network for weakly supervised segmentation |
| title_fullStr | Cross-image region mining with region prototypical network for weakly supervised segmentation |
| title_full_unstemmed | Cross-image region mining with region prototypical network for weakly supervised segmentation |
| title_short | Cross-image region mining with region prototypical network for weakly supervised segmentation |
| title_sort | cross image region mining with region prototypical network for weakly supervised segmentation |
| topic | Engineering::Computer science and engineering Cross-Image Segmentation |
| url | https://hdl.handle.net/10356/162959 |
| work_keys_str_mv | AT liuweide crossimageregionminingwithregionprototypicalnetworkforweaklysupervisedsegmentation AT kongxiangfei crossimageregionminingwithregionprototypicalnetworkforweaklysupervisedsegmentation AT hungtzuyi crossimageregionminingwithregionprototypicalnetworkforweaklysupervisedsegmentation AT linguosheng crossimageregionminingwithregionprototypicalnetworkforweaklysupervisedsegmentation |