Revisiting deep structured models for pixel-level labeling with gradient-based inference
Semantic segmentation and other pixel-level labeling tasks have made significant progress recently due to the deep learning paradigm. Many state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors and label consist...
| Main Authors: | , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Society for Industrial & Applied Mathematics
2018
|
| _version_ | 1797051716611342336 |
|---|---|
| author | Larsson, M Arnab, A Zheng, S Torr, P Kahl, F |
| author_facet | Larsson, M Arnab, A Zheng, S Torr, P Kahl, F |
| author_sort | Larsson, M |
| collection | OXFORD |
| description | Semantic segmentation and other pixel-level labeling tasks have made significant progress recently due to the deep learning paradigm. Many state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors and label consistencies and feature-based image conditioning. These random field models with image conditioning typically require computationally demanding filtering techniques during inference. In this paper, we present a new inference and learning framework which can learn arbitrary pairwise conditional random field (CRF) potentials. Both standard spatial and high-dimensional bilateral kernels are considered. In addition, we introduce a new type of potential function which is image-dependent like the bilateral kernel, but an order of magnitude faster to compute since only spatial convolutions are employed. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label-class interactions are indeed better modeled by a non-Gaussian potential. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models. |
| first_indexed | 2024-03-06T18:23:26Z |
| format | Journal article |
| id | oxford-uuid:07172dfd-e81e-435c-a810-1b193de2734d |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T18:23:26Z |
| publishDate | 2018 |
| publisher | Society for Industrial & Applied Mathematics |
| record_format | dspace |
| spelling | oxford-uuid:07172dfd-e81e-435c-a810-1b193de2734d2022-03-26T09:05:57ZRevisiting deep structured models for pixel-level labeling with gradient-based inferenceJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:07172dfd-e81e-435c-a810-1b193de2734dEnglishSymplectic Elements at OxfordSociety for Industrial & Applied Mathematics2018Larsson, MArnab, AZheng, STorr, PKahl, FSemantic segmentation and other pixel-level labeling tasks have made significant progress recently due to the deep learning paradigm. Many state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors and label consistencies and feature-based image conditioning. These random field models with image conditioning typically require computationally demanding filtering techniques during inference. In this paper, we present a new inference and learning framework which can learn arbitrary pairwise conditional random field (CRF) potentials. Both standard spatial and high-dimensional bilateral kernels are considered. In addition, we introduce a new type of potential function which is image-dependent like the bilateral kernel, but an order of magnitude faster to compute since only spatial convolutions are employed. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label-class interactions are indeed better modeled by a non-Gaussian potential. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models. |
| spellingShingle | Larsson, M Arnab, A Zheng, S Torr, P Kahl, F Revisiting deep structured models for pixel-level labeling with gradient-based inference |
| title | Revisiting deep structured models for pixel-level labeling with gradient-based inference |
| title_full | Revisiting deep structured models for pixel-level labeling with gradient-based inference |
| title_fullStr | Revisiting deep structured models for pixel-level labeling with gradient-based inference |
| title_full_unstemmed | Revisiting deep structured models for pixel-level labeling with gradient-based inference |
| title_short | Revisiting deep structured models for pixel-level labeling with gradient-based inference |
| title_sort | revisiting deep structured models for pixel level labeling with gradient based inference |
| work_keys_str_mv | AT larssonm revisitingdeepstructuredmodelsforpixellevellabelingwithgradientbasedinference AT arnaba revisitingdeepstructuredmodelsforpixellevellabelingwithgradientbasedinference AT zhengs revisitingdeepstructuredmodelsforpixellevellabelingwithgradientbasedinference AT torrp revisitingdeepstructuredmodelsforpixellevellabelingwithgradientbasedinference AT kahlf revisitingdeepstructuredmodelsforpixellevellabelingwithgradientbasedinference |