Supervising the new with the old: Learning SFM from SFM
Recent work has demonstrated that it is possible to learn deep neural networks for monocular depth and ego-motion estimation from unlabelled video sequences, an interesting theoretical development with numerous advantages in applications. In this paper, we propose a number of improvements to these a...
| Main Authors: | , |
|---|---|
| Format: | Conference item |
| Published: |
Springer
2018
|
| _version_ | 1797065736277983232 |
|---|---|
| author | Klodt, M Vedaldi, A |
| author_facet | Klodt, M Vedaldi, A |
| author_sort | Klodt, M |
| collection | OXFORD |
| description | Recent work has demonstrated that it is possible to learn deep neural networks for monocular depth and ego-motion estimation from unlabelled video sequences, an interesting theoretical development with numerous advantages in applications. In this paper, we propose a number of improvements to these approaches. First, since such self-supervised approaches are based on the brightness constancy assumption, which is valid only for a subset of pixels, we propose a probabilistic learning formulation where the network predicts distributions over variables rather than specific values. As these distributions are conditioned on the observed image, the network can learn which scene and object types are likely to violate the model assumptions, resulting in more robust learning. We also propose to build on dozens of years of experience in developing handcrafted structure-from-motion (SFM) algorithms. We do so by using an off-the-shelf SFM system to generate a supervisory signal for the deep neural network. While this signal is also noisy, we show that our probabilistic formulation can learn and account for the defects of SFM, helping to integrate different sources of information and boosting the overall performance of the network. |
| first_indexed | 2024-03-06T21:32:50Z |
| format | Conference item |
| id | oxford-uuid:45401dc3-44cb-4e74-a589-16937f5c3450 |
| institution | University of Oxford |
| last_indexed | 2024-03-06T21:32:50Z |
| publishDate | 2018 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:45401dc3-44cb-4e74-a589-16937f5c34502022-03-26T15:06:41ZSupervising the new with the old: Learning SFM from SFMConference itemhttp://purl.org/coar/resource_type/c_5794uuid:45401dc3-44cb-4e74-a589-16937f5c3450Symplectic Elements at OxfordSpringer2018Klodt, MVedaldi, ARecent work has demonstrated that it is possible to learn deep neural networks for monocular depth and ego-motion estimation from unlabelled video sequences, an interesting theoretical development with numerous advantages in applications. In this paper, we propose a number of improvements to these approaches. First, since such self-supervised approaches are based on the brightness constancy assumption, which is valid only for a subset of pixels, we propose a probabilistic learning formulation where the network predicts distributions over variables rather than specific values. As these distributions are conditioned on the observed image, the network can learn which scene and object types are likely to violate the model assumptions, resulting in more robust learning. We also propose to build on dozens of years of experience in developing handcrafted structure-from-motion (SFM) algorithms. We do so by using an off-the-shelf SFM system to generate a supervisory signal for the deep neural network. While this signal is also noisy, we show that our probabilistic formulation can learn and account for the defects of SFM, helping to integrate different sources of information and boosting the overall performance of the network. |
| spellingShingle | Klodt, M Vedaldi, A Supervising the new with the old: Learning SFM from SFM |
| title | Supervising the new with the old: Learning SFM from SFM |
| title_full | Supervising the new with the old: Learning SFM from SFM |
| title_fullStr | Supervising the new with the old: Learning SFM from SFM |
| title_full_unstemmed | Supervising the new with the old: Learning SFM from SFM |
| title_short | Supervising the new with the old: Learning SFM from SFM |
| title_sort | supervising the new with the old learning sfm from sfm |
| work_keys_str_mv | AT klodtm supervisingthenewwiththeoldlearningsfmfromsfm AT vedaldia supervisingthenewwiththeoldlearningsfmfromsfm |