Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation
We propose a weakly supervised approach to semantic segmentation using bounding box annotations. Bounding boxes are treated as noisy labels for the foreground objects. We predict a per-class attention map that saliently guides the per-pixel cross entropy loss to focus on foreground pixels and refine...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Springer International Publishing
2020
|
| _version_ | 1797079170207973376 |
|---|---|
| author | Kulharia, V Chandra, S Agrawal, A Torr, PHS Tyagi, A |
| author_facet | Kulharia, V Chandra, S Agrawal, A Torr, PHS Tyagi, A |
| author_sort | Kulharia, V |
| collection | OXFORD |
| description | We propose a weakly supervised approach to semantic segmentation using bounding box annotations. Bounding boxes are treated as noisy labels for the foreground objects. We predict a per-class attention map that saliently guides the per-pixel cross entropy loss to focus on foreground pixels and refines the segmentation boundaries. This avoids propagating erroneous gradients due to incorrect foreground labels on the background. Additionally, we learn pixel embeddings to simultaneously optimize for high intra-class feature affinity while increasing discrimination between features across different classes. Our method, Box2Seg, achieves state-of-the-art segmentation accuracy on PASCAL VOC 2012 by significantly improving the mIOU metric by 2.1% compared to previous weakly supervised approaches. Our weakly supervised approach is comparable to the recent fully supervised methods when fine-tuned with limited amount of pixel-level annotations. Qualitative results and ablation studies show the benefit of different loss terms on the overall performance. |
| first_indexed | 2024-03-07T00:41:55Z |
| format | Conference item |
| id | oxford-uuid:83565a76-c0cf-44b8-b99c-30e8b33721a9 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T00:41:55Z |
| publishDate | 2020 |
| publisher | Springer International Publishing |
| record_format | dspace |
| spelling | oxford-uuid:83565a76-c0cf-44b8-b99c-30e8b33721a92022-03-26T21:43:30ZBox2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentationConference itemhttp://purl.org/coar/resource_type/c_5794uuid:83565a76-c0cf-44b8-b99c-30e8b33721a9EnglishSymplectic ElementsSpringer International Publishing2020Kulharia, VChandra, SAgrawal, ATorr, PHSTyagi, AWe propose a weakly supervised approach to semantic segmentation using bounding box annotations. Bounding boxes are treated as noisy labels for the foreground objects. We predict a per-class attention map that saliently guides the per-pixel cross entropy loss to focus on foreground pixels and refines the segmentation boundaries. This avoids propagating erroneous gradients due to incorrect foreground labels on the background. Additionally, we learn pixel embeddings to simultaneously optimize for high intra-class feature affinity while increasing discrimination between features across different classes. Our method, Box2Seg, achieves state-of-the-art segmentation accuracy on PASCAL VOC 2012 by significantly improving the mIOU metric by 2.1% compared to previous weakly supervised approaches. Our weakly supervised approach is comparable to the recent fully supervised methods when fine-tuned with limited amount of pixel-level annotations. Qualitative results and ablation studies show the benefit of different loss terms on the overall performance. |
| spellingShingle | Kulharia, V Chandra, S Agrawal, A Torr, PHS Tyagi, A Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| title | Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| title_full | Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| title_fullStr | Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| title_full_unstemmed | Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| title_short | Box2Seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| title_sort | box2seg attention weighted loss and discriminative feature learning for weakly supervised segmentation |
| work_keys_str_mv | AT kulhariav box2segattentionweightedlossanddiscriminativefeaturelearningforweaklysupervisedsegmentation AT chandras box2segattentionweightedlossanddiscriminativefeaturelearningforweaklysupervisedsegmentation AT agrawala box2segattentionweightedlossanddiscriminativefeaturelearningforweaklysupervisedsegmentation AT torrphs box2segattentionweightedlossanddiscriminativefeaturelearningforweaklysupervisedsegmentation AT tyagia box2segattentionweightedlossanddiscriminativefeaturelearningforweaklysupervisedsegmentation |