A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials
<p>Are we using the right potential functions in the Conditional Random Field models that are popular in the Vision community? Semantic segmentation and other pixel-level labelling tasks have made significant progress recently due to the deep learning paradigm. However, most state-of-the-art s...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference item |
| Published: |
Springer
2018
|
| _version_ | 1797070831438790656 |
|---|---|
| author | Larsson, M Arnab, A Kahl, F Zheng, S Torr, P |
| author_facet | Larsson, M Arnab, A Kahl, F Zheng, S Torr, P |
| author_sort | Larsson, M |
| collection | OXFORD |
| description | <p>Are we using the right potential functions in the Conditional Random Field models that are popular in the Vision community? Semantic segmentation and other pixel-level labelling tasks have made significant progress recently due to the deep learning paradigm. However, most state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors, label consistencies and feature-based image conditioning.</p> <br/> <p>In this paper, we challenge this view by developing a new inference and learning framework which can learn pairwise CRF potentials restricted only by their dependence on the image pixel values and the size of the support. Both standard spatial and high-dimensional bilateral kernels are considered. Our framework is based on the observation that CRF inference can be achieved via projected gradient descent and consequently, can easily be integrated in deep neural networks to allow for end-to-end training. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential. In addition, we compare our inference method to the commonly used mean-field algorithm. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models.</p> |
| first_indexed | 2024-03-06T22:44:37Z |
| format | Conference item |
| id | oxford-uuid:5cbdbaa8-b1e8-4d2c-99dd-616ec359cf09 |
| institution | University of Oxford |
| last_indexed | 2024-03-06T22:44:37Z |
| publishDate | 2018 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:5cbdbaa8-b1e8-4d2c-99dd-616ec359cf092022-03-26T17:30:04ZA projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentialsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:5cbdbaa8-b1e8-4d2c-99dd-616ec359cf09Symplectic Elements at OxfordSpringer2018Larsson, MArnab, AKahl, FZheng, STorr, P<p>Are we using the right potential functions in the Conditional Random Field models that are popular in the Vision community? Semantic segmentation and other pixel-level labelling tasks have made significant progress recently due to the deep learning paradigm. However, most state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors, label consistencies and feature-based image conditioning.</p> <br/> <p>In this paper, we challenge this view by developing a new inference and learning framework which can learn pairwise CRF potentials restricted only by their dependence on the image pixel values and the size of the support. Both standard spatial and high-dimensional bilateral kernels are considered. Our framework is based on the observation that CRF inference can be achieved via projected gradient descent and consequently, can easily be integrated in deep neural networks to allow for end-to-end training. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential. In addition, we compare our inference method to the commonly used mean-field algorithm. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models.</p> |
| spellingShingle | Larsson, M Arnab, A Kahl, F Zheng, S Torr, P A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials |
| title | A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials |
| title_full | A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials |
| title_fullStr | A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials |
| title_full_unstemmed | A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials |
| title_short | A projected gradient descent method for CRF inference allowing end-to-end training of arbitrary pairwise potentials |
| title_sort | projected gradient descent method for crf inference allowing end to end training of arbitrary pairwise potentials |
| work_keys_str_mv | AT larssonm aprojectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT arnaba aprojectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT kahlf aprojectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT zhengs aprojectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT torrp aprojectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT larssonm projectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT arnaba projectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT kahlf projectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT zhengs projectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials AT torrp projectedgradientdescentmethodforcrfinferenceallowingendtoendtrainingofarbitrarypairwisepotentials |