Self-Supervised Depth Completion Based on Multi-Modal Spatio-Temporal Consistency

Due to the low cost and easy deployment, self-supervised depth completion has been widely studied in recent years. In this work, a self-supervised depth completion method is designed based on multi-modal spatio-temporal consistency (MSC). The self-supervised depth completion nowadays faces other pro...

Full description

Bibliographic Details
Main Authors: Quan Zhang, Xiaoyu Chen, Xingguo Wang, Jing Han, Yi Zhang, Jiang Yue
Format: Article
Language:English
Published: MDPI AG 2022-12-01
Series:Remote Sensing
Subjects:
Online Access:https://www.mdpi.com/2072-4292/15/1/135
Description
Summary:Due to the low cost and easy deployment, self-supervised depth completion has been widely studied in recent years. In this work, a self-supervised depth completion method is designed based on multi-modal spatio-temporal consistency (MSC). The self-supervised depth completion nowadays faces other problems: moving objects, occluded/dark light/low texture parts, long-distance completion, and cross-modal fusion. In the face of these problems, the most critical novelty of this work lies in that the self-supervised mechanism is designed to train the depth completion network by MSC constraint. It not only makes better use of depth-temporal data, but also plays the advantage of photometric-temporal constraint. With the self-supervised mechanism of MSC constraint, the overall system outperforms many other self-supervised networks, even exceeding partially supervised networks.
ISSN:2072-4292