Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints
This paper proposes a novel unsupervised learning framework for depth recovery and camera ego-motion estimation from monocular video. The framework exploits the optical flow (OF) property to jointly train the depth and the ego-motion models. Unlike the existing unsupervised methods, our method extra...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/4/1383 |
| _version_ | 1797476763057520640 |
|---|---|
| author | Baigan Zhao Yingping Huang Wenyan Ci Xing Hu |
| author_facet | Baigan Zhao Yingping Huang Wenyan Ci Xing Hu |
| author_sort | Baigan Zhao |
| collection | DOAJ |
| description | This paper proposes a novel unsupervised learning framework for depth recovery and camera ego-motion estimation from monocular video. The framework exploits the optical flow (OF) property to jointly train the depth and the ego-motion models. Unlike the existing unsupervised methods, our method extracts the features from the optical flow rather than from the raw RGB images, thereby enhancing unsupervised learning. In addition, we exploit the forward-backward consistency check of the optical flow to generate a mask of the invalid region in the image, and accordingly, eliminate the outlier regions such as occlusion regions and moving objects for the learning. Furthermore, in addition to using view synthesis as a supervised signal, we impose additional loss functions, including optical flow consistency loss and depth consistency loss, as additional supervision signals on the valid image region to further enhance the training of the models. Substantial experiments on multiple benchmark datasets demonstrate that our method outperforms other unsupervised methods. |
| first_indexed | 2024-03-09T21:07:15Z |
| format | Article |
| id | doaj.art-87ca35686baf40349a9916a4ec57a05f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T21:07:15Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-87ca35686baf40349a9916a4ec57a05f2023-11-23T21:58:39ZengMDPI AGSensors1424-82202022-02-01224138310.3390/s22041383Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple ConstraintsBaigan Zhao0Yingping Huang1Wenyan Ci2Xing Hu3School of Optical-Electrical and Computer Engineering, University of Shanghai for Science & Technology, Shanghai 200093, ChinaSchool of Optical-Electrical and Computer Engineering, University of Shanghai for Science & Technology, Shanghai 200093, ChinaSchool of Engineering, Huzhou University, Huzhou 313000, ChinaSchool of Optical-Electrical and Computer Engineering, University of Shanghai for Science & Technology, Shanghai 200093, ChinaThis paper proposes a novel unsupervised learning framework for depth recovery and camera ego-motion estimation from monocular video. The framework exploits the optical flow (OF) property to jointly train the depth and the ego-motion models. Unlike the existing unsupervised methods, our method extracts the features from the optical flow rather than from the raw RGB images, thereby enhancing unsupervised learning. In addition, we exploit the forward-backward consistency check of the optical flow to generate a mask of the invalid region in the image, and accordingly, eliminate the outlier regions such as occlusion regions and moving objects for the learning. Furthermore, in addition to using view synthesis as a supervised signal, we impose additional loss functions, including optical flow consistency loss and depth consistency loss, as additional supervision signals on the valid image region to further enhance the training of the models. Substantial experiments on multiple benchmark datasets demonstrate that our method outperforms other unsupervised methods.https://www.mdpi.com/1424-8220/22/4/1383unsupervised learningdepth recoveryego-motion estimationoptical flow |
| spellingShingle | Baigan Zhao Yingping Huang Wenyan Ci Xing Hu Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints Sensors unsupervised learning depth recovery ego-motion estimation optical flow |
| title | Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints |
| title_full | Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints |
| title_fullStr | Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints |
| title_full_unstemmed | Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints |
| title_short | Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints |
| title_sort | unsupervised learning of monocular depth and ego motion with optical flow features and multiple constraints |
| topic | unsupervised learning depth recovery ego-motion estimation optical flow |
| url | https://www.mdpi.com/1424-8220/22/4/1383 |
| work_keys_str_mv | AT baiganzhao unsupervisedlearningofmonoculardepthandegomotionwithopticalflowfeaturesandmultipleconstraints AT yingpinghuang unsupervisedlearningofmonoculardepthandegomotionwithopticalflowfeaturesandmultipleconstraints AT wenyanci unsupervisedlearningofmonoculardepthandegomotionwithopticalflowfeaturesandmultipleconstraints AT xinghu unsupervisedlearningofmonoculardepthandegomotionwithopticalflowfeaturesandmultipleconstraints |