Regularization for Unsupervised Learning of Optical Flow
Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convol...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/8/4080 |
| _version_ | 1797603488459390976 |
|---|---|
| author | Libo Long Jochen Lang |
| author_facet | Libo Long Jochen Lang |
| author_sort | Libo Long |
| collection | DOAJ |
| description | Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net. |
| first_indexed | 2024-03-11T04:32:50Z |
| format | Article |
| id | doaj.art-3f6b7226f7864cb2bac27377618e41be |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T04:32:50Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-3f6b7226f7864cb2bac27377618e41be2023-11-17T21:18:46ZengMDPI AGSensors1424-82202023-04-01238408010.3390/s23084080Regularization for Unsupervised Learning of Optical FlowLibo Long0Jochen Lang1Faculty of Engineering, University of Ottawa, Ottawa, ON K1N 6N5, CanadaFaculty of Engineering, University of Ottawa, Ottawa, ON K1N 6N5, CanadaRegularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net.https://www.mdpi.com/1424-8220/23/8/4080self-supervised trainingteacher–student learningregularizationoptical flowscene flow |
| spellingShingle | Libo Long Jochen Lang Regularization for Unsupervised Learning of Optical Flow Sensors self-supervised training teacher–student learning regularization optical flow scene flow |
| title | Regularization for Unsupervised Learning of Optical Flow |
| title_full | Regularization for Unsupervised Learning of Optical Flow |
| title_fullStr | Regularization for Unsupervised Learning of Optical Flow |
| title_full_unstemmed | Regularization for Unsupervised Learning of Optical Flow |
| title_short | Regularization for Unsupervised Learning of Optical Flow |
| title_sort | regularization for unsupervised learning of optical flow |
| topic | self-supervised training teacher–student learning regularization optical flow scene flow |
| url | https://www.mdpi.com/1424-8220/23/8/4080 |
| work_keys_str_mv | AT libolong regularizationforunsupervisedlearningofopticalflow AT jochenlang regularizationforunsupervisedlearningofopticalflow |