Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry
Abstract The external calibration between 3D LiDAR and 2D camera is an extremely important step towards multimodal fusion for robot perception. However, its accuracy is still unsatisfactory. To improve the accuracy of calibration, we first analyze the interference factors that affect the performance...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2023-06-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-023-01140-1 |
| _version_ | 1797647157241577472 |
|---|---|
| author | Ruyu Liu Jieying Shi Haoyu Zhang Jianhua Zhang Bo Sun |
| author_facet | Ruyu Liu Jieying Shi Haoyu Zhang Jianhua Zhang Bo Sun |
| author_sort | Ruyu Liu |
| collection | DOAJ |
| description | Abstract The external calibration between 3D LiDAR and 2D camera is an extremely important step towards multimodal fusion for robot perception. However, its accuracy is still unsatisfactory. To improve the accuracy of calibration, we first analyze the interference factors that affect the performance of the calibration model under a causal inference framework in this study. Guided by the causality analysis, we present Iter-CalibNet (Iterative Calibration Convolutional Neural Network) to infer a 6 degrees of freedom (DoF) rigid body transformation between 3D LiDAR and 2D camera. By downscaling point clouds to obtain more overlapping region between 3D–2D data pair and applying iterative calibration manner, the interference of confounding bias in the calibration model is effectively eliminated. Moreover, our Iter-CalibNet adds non-local neural network after each convolution operation to capture the transformation relationship. We also combine the geometric loss and photometric loss obtained from the interframe constraints to optimize the calibration accuracy. Extensive experiments demonstrate that our Iter-CalibNet can achieve leading performance by comparison with other CNN based and traditional calibration methods. |
| first_indexed | 2024-03-11T15:12:18Z |
| format | Article |
| id | doaj.art-fa5bdf62b73a4b5995da3393c5b5fe70 |
| institution | Directory Open Access Journal |
| issn | 2199-4536 2198-6053 |
| language | English |
| last_indexed | 2024-03-11T15:12:18Z |
| publishDate | 2023-06-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj.art-fa5bdf62b73a4b5995da3393c5b5fe702023-10-29T12:41:17ZengSpringerComplex & Intelligent Systems2199-45362198-60532023-06-01967349736310.1007/s40747-023-01140-1Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometryRuyu Liu0Jieying Shi1Haoyu Zhang2Jianhua Zhang3Bo Sun4School of Information Science and Technology, Hangzhou Normal UniversityThe College of Computer Science, Zhejiang University of TechnologySchool of Information Science and Technology, Hangzhou Normal UniversitySchool of Computer Science and Engineering, Tianjin University of TechnologyHaixi Institutes, Chinese Academy of Sciences Quanzhou Institute of Equipment ManufacturingAbstract The external calibration between 3D LiDAR and 2D camera is an extremely important step towards multimodal fusion for robot perception. However, its accuracy is still unsatisfactory. To improve the accuracy of calibration, we first analyze the interference factors that affect the performance of the calibration model under a causal inference framework in this study. Guided by the causality analysis, we present Iter-CalibNet (Iterative Calibration Convolutional Neural Network) to infer a 6 degrees of freedom (DoF) rigid body transformation between 3D LiDAR and 2D camera. By downscaling point clouds to obtain more overlapping region between 3D–2D data pair and applying iterative calibration manner, the interference of confounding bias in the calibration model is effectively eliminated. Moreover, our Iter-CalibNet adds non-local neural network after each convolution operation to capture the transformation relationship. We also combine the geometric loss and photometric loss obtained from the interframe constraints to optimize the calibration accuracy. Extensive experiments demonstrate that our Iter-CalibNet can achieve leading performance by comparison with other CNN based and traditional calibration methods.https://doi.org/10.1007/s40747-023-01140-1Causality analysisLiDAR and camera calibrationDeep learningGeometryMachine vision |
| spellingShingle | Ruyu Liu Jieying Shi Haoyu Zhang Jianhua Zhang Bo Sun Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry Complex & Intelligent Systems Causality analysis LiDAR and camera calibration Deep learning Geometry Machine vision |
| title | Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry |
| title_full | Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry |
| title_fullStr | Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry |
| title_full_unstemmed | Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry |
| title_short | Causal calibration: iteratively calibrating LiDAR and camera by considering causality and geometry |
| title_sort | causal calibration iteratively calibrating lidar and camera by considering causality and geometry |
| topic | Causality analysis LiDAR and camera calibration Deep learning Geometry Machine vision |
| url | https://doi.org/10.1007/s40747-023-01140-1 |
| work_keys_str_mv | AT ruyuliu causalcalibrationiterativelycalibratinglidarandcamerabyconsideringcausalityandgeometry AT jieyingshi causalcalibrationiterativelycalibratinglidarandcamerabyconsideringcausalityandgeometry AT haoyuzhang causalcalibrationiterativelycalibratinglidarandcamerabyconsideringcausalityandgeometry AT jianhuazhang causalcalibrationiterativelycalibratinglidarandcamerabyconsideringcausalityandgeometry AT bosun causalcalibrationiterativelycalibratinglidarandcamerabyconsideringcausalityandgeometry |