WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation
Weakly supervised instance segmentation (WSIS) provides a promising way to address instance segmentation in the absence of sufficient labeled data for training. Previous attempts on WSIS usually follow a proposal-based paradigm, critical to which is the proposal scoring strategy. These works mostly...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-05-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/10/3475 |
| _version_ | 1797533886818811904 |
|---|---|
| author | Jia-Rong Ou Shu-Le Deng Jin-Gang Yu |
| author_facet | Jia-Rong Ou Shu-Le Deng Jin-Gang Yu |
| author_sort | Jia-Rong Ou |
| collection | DOAJ |
| description | Weakly supervised instance segmentation (WSIS) provides a promising way to address instance segmentation in the absence of sufficient labeled data for training. Previous attempts on WSIS usually follow a proposal-based paradigm, critical to which is the proposal scoring strategy. These works mostly rely on certain heuristic strategies for proposal scoring, which largely hampers the sustainable advances concerning WSIS. Towards this end, this paper introduces a novel framework for weakly supervised instance segmentation, called Weakly Supervised R-CNN (WS-RCNN). The basic idea is to deploy a deep network to learn to score proposals, under the special setting of weak supervision. To tackle the key issue of acquiring proposal-level pseudo labels for model training, we propose a so-called Attention-Guided Pseudo Labeling (AGPL) strategy, which leverages the local maximal (peaks) in image-level attention maps and the spatial relationship among peaks and proposals to infer pseudo labels. We also suggest a novel training loss, called Entropic OpenSet Loss, to handle background proposals more effectively so as to further improve the robustness. Comprehensive experiments on two standard benchmarking datasets demonstrate that the proposed WS-RCNN can outperform the state-of-the-art by a large margin, with an improvement of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>11.6</mn><mo>%</mo></mrow></semantics></math></inline-formula> on PASCAL VOC 2012 and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>10.7</mn><mo>%</mo></mrow></semantics></math></inline-formula> on MS COCO 2014 in terms of mAP<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mrow></mrow><mn>50</mn></msub></semantics></math></inline-formula>, which indicates that learning-based proposal scoring and the proposed WS-RCNN framework might be a promising way towards WSIS. |
| first_indexed | 2024-03-10T11:21:05Z |
| format | Article |
| id | doaj.art-7ca2ba1c53e84460a33b9de445c32c9f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T11:21:05Z |
| publishDate | 2021-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-7ca2ba1c53e84460a33b9de445c32c9f2023-11-21T20:00:42ZengMDPI AGSensors1424-82202021-05-012110347510.3390/s21103475WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance SegmentationJia-Rong Ou0Shu-Le Deng1Jin-Gang Yu2School of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, ChinaSchool of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, ChinaSchool of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, ChinaWeakly supervised instance segmentation (WSIS) provides a promising way to address instance segmentation in the absence of sufficient labeled data for training. Previous attempts on WSIS usually follow a proposal-based paradigm, critical to which is the proposal scoring strategy. These works mostly rely on certain heuristic strategies for proposal scoring, which largely hampers the sustainable advances concerning WSIS. Towards this end, this paper introduces a novel framework for weakly supervised instance segmentation, called Weakly Supervised R-CNN (WS-RCNN). The basic idea is to deploy a deep network to learn to score proposals, under the special setting of weak supervision. To tackle the key issue of acquiring proposal-level pseudo labels for model training, we propose a so-called Attention-Guided Pseudo Labeling (AGPL) strategy, which leverages the local maximal (peaks) in image-level attention maps and the spatial relationship among peaks and proposals to infer pseudo labels. We also suggest a novel training loss, called Entropic OpenSet Loss, to handle background proposals more effectively so as to further improve the robustness. Comprehensive experiments on two standard benchmarking datasets demonstrate that the proposed WS-RCNN can outperform the state-of-the-art by a large margin, with an improvement of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>11.6</mn><mo>%</mo></mrow></semantics></math></inline-formula> on PASCAL VOC 2012 and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>10.7</mn><mo>%</mo></mrow></semantics></math></inline-formula> on MS COCO 2014 in terms of mAP<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mrow></mrow><mn>50</mn></msub></semantics></math></inline-formula>, which indicates that learning-based proposal scoring and the proposed WS-RCNN framework might be a promising way towards WSIS.https://www.mdpi.com/1424-8220/21/10/3475weakly supervised learninginstance segmentationproposal scoring network |
| spellingShingle | Jia-Rong Ou Shu-Le Deng Jin-Gang Yu WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation Sensors weakly supervised learning instance segmentation proposal scoring network |
| title | WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation |
| title_full | WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation |
| title_fullStr | WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation |
| title_full_unstemmed | WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation |
| title_short | WS-RCNN: Learning to Score Proposals for Weakly Supervised Instance Segmentation |
| title_sort | ws rcnn learning to score proposals for weakly supervised instance segmentation |
| topic | weakly supervised learning instance segmentation proposal scoring network |
| url | https://www.mdpi.com/1424-8220/21/10/3475 |
| work_keys_str_mv | AT jiarongou wsrcnnlearningtoscoreproposalsforweaklysupervisedinstancesegmentation AT shuledeng wsrcnnlearningtoscoreproposalsforweaklysupervisedinstancesegmentation AT jingangyu wsrcnnlearningtoscoreproposalsforweaklysupervisedinstancesegmentation |