LiDAR and radar based localization with deep neural networks - Wang, W (2022)
Localization is a fundamental problem in many application scenarios, including robotics, computer vision and autonomous driving. In this thesis, localization is defined as two sub-tasks: odometry and relocalization. Odometry estimates the relative poses of a sensor in real-time when provided with a sequential series of sensory input data, and relocalization predicts the global poses of a sensor from previously traversed scenes. Recently, deep neural networks have been applied to visual localization and shown promising results. However, vision sensors such as cameras are greatly impacted by environmental conditions which limits the application of purely visual methods in the real world. Active sensors, such as LiDAR and radar, can capture rich geometric and semantic information in the forms of point clouds and polar scans and they are significantly more robust to environmental variations. Inspired by the recent deep visual localization methods, this thesis studies how to apply deep learning for the localization task using LiDAR and radar sensors. Specifically, this thesis studies three aspects of LiDAR and radar based localization with deep neural networks. First, an end-to-end parallel neural network termed DeepPCO is proposed for the LiDAR odometry task. It consists of two sub-networks to predict translation and orientation respectively. Second, a novel LiDAR point cloud-based method for deep LiDAR relocalization is introduced, termed PointLoc. It learns features from point clouds directly for estimating global poses in an end-to-end manner. In addition, a new indoor LiDAR-visual dataset termed vReLoc is released for studying the LiDAR relocalization problem. Third, a radar-based relocalization system, RadarLoc, is proposed by utilizing deep learning and FMCW radar scans. It designs a selfattention module with a nested encoder-decoder architecture to mask out outliers. Geometric constraints are leveraged in the loss function to improve the overall performance. Extensive experiments on large-scale datasets have been conducted to demonstrate the effectiveness of the above three approaches. This thesis also discusses implications and future research directions of this field.