Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization
Unsupervised localization and segmentation are long-standing computer vision challenges that involve decom-posing an image into semantically meaningful segments without any labeled data. These tasks are particularly interesting in an unsupervised setting due to the difficulty and cost of obtaining d...
| मुख्य लेखकों: | , , , |
|---|---|
| स्वरूप: | Conference item |
| भाषा: | English |
| प्रकाशित: |
IEEE
2022
|
| _version_ | 1826308523159453696 |
|---|---|
| author | Melas-Kyriazi, L Rupprecht, C Laina, I Vedaldi, A |
| author_facet | Melas-Kyriazi, L Rupprecht, C Laina, I Vedaldi, A |
| author_sort | Melas-Kyriazi, L |
| collection | OXFORD |
| description | Unsupervised localization and segmentation are long-standing computer vision challenges that involve decom-posing an image into semantically meaningful segments without any labeled data. These tasks are particularly interesting in an unsupervised setting due to the difficulty and cost of obtaining dense image annotations, but existing un-supervised approaches struggle with complex scenes containing multiple objects. Differently from existing methods, which are purely based on deep learning, we take inspiration from traditional spectral segmentation methods by re-framing image decomposition as a graph partitioning problem. Specifically, we examine the eigenvectors of the Laplacian of a feature affinity matrix from self-supervised networks. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. Furthermore, by clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions, i.e. semantic segmentations. Experiments on complex datasets (PASCAL VOC, MS-COCO) demonstrate that our simple spectral method outperforms the state-of-the-art in unsupervised localization and segmentation by a significant margin. Furthermore, our method can be readily usedfor a variety of complex image editing tasks, such as background removal and compositing. 1 1 Project Page: https://lukemelas.github.io/deep-spectral-segmentation/ |
| first_indexed | 2024-03-07T07:20:42Z |
| format | Conference item |
| id | oxford-uuid:013da010-b099-4c6c-a99c-b73fa0ae77fc |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:20:42Z |
| publishDate | 2022 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:013da010-b099-4c6c-a99c-b73fa0ae77fc2022-10-10T10:01:31ZDeep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localizationConference itemhttp://purl.org/coar/resource_type/c_5794uuid:013da010-b099-4c6c-a99c-b73fa0ae77fcEnglishSymplectic ElementsIEEE2022Melas-Kyriazi, LRupprecht, CLaina, IVedaldi, AUnsupervised localization and segmentation are long-standing computer vision challenges that involve decom-posing an image into semantically meaningful segments without any labeled data. These tasks are particularly interesting in an unsupervised setting due to the difficulty and cost of obtaining dense image annotations, but existing un-supervised approaches struggle with complex scenes containing multiple objects. Differently from existing methods, which are purely based on deep learning, we take inspiration from traditional spectral segmentation methods by re-framing image decomposition as a graph partitioning problem. Specifically, we examine the eigenvectors of the Laplacian of a feature affinity matrix from self-supervised networks. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. Furthermore, by clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions, i.e. semantic segmentations. Experiments on complex datasets (PASCAL VOC, MS-COCO) demonstrate that our simple spectral method outperforms the state-of-the-art in unsupervised localization and segmentation by a significant margin. Furthermore, our method can be readily usedfor a variety of complex image editing tasks, such as background removal and compositing. 1 1 Project Page: https://lukemelas.github.io/deep-spectral-segmentation/ |
| spellingShingle | Melas-Kyriazi, L Rupprecht, C Laina, I Vedaldi, A Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| title | Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| title_full | Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| title_fullStr | Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| title_full_unstemmed | Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| title_short | Deep spectral methods: a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| title_sort | deep spectral methods a surprisingly strong baseline for unsupervised semantic segmentation and localization |
| work_keys_str_mv | AT melaskyriazil deepspectralmethodsasurprisinglystrongbaselineforunsupervisedsemanticsegmentationandlocalization AT rupprechtc deepspectralmethodsasurprisinglystrongbaselineforunsupervisedsemanticsegmentationandlocalization AT lainai deepspectralmethodsasurprisinglystrongbaselineforunsupervisedsemanticsegmentationandlocalization AT vedaldia deepspectralmethodsasurprisinglystrongbaselineforunsupervisedsemanticsegmentationandlocalization |