SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes
Rigorous boresight calibration between light detection and ranging (LiDAR) and the camera is crucial for geometry and optical information fusion in Earth observation and robotic applications. Although boresight parameters can be obtained through precalibration with artificial targets, unforeseen mov...
| Main Authors: | , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/169994 |
| _version_ | 1826110817718763520 |
|---|---|
| author | Liao, Youqi Li, Jianping Kang, Shuhao Li, Qiang Zhu, Guifang Yuan, Shenghai Dong, Zhen Yang, Bisheng |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Liao, Youqi Li, Jianping Kang, Shuhao Li, Qiang Zhu, Guifang Yuan, Shenghai Dong, Zhen Yang, Bisheng |
| author_sort | Liao, Youqi |
| collection | NTU |
| description | Rigorous boresight calibration between light detection and ranging (LiDAR) and the camera is crucial for geometry and optical information fusion in Earth observation and robotic applications. Although boresight parameters can be obtained through precalibration with artificial targets, unforeseen movement of sensors during data collection can lead to significant errors in the boresight parameters. To address this issue, we propose SE-Calib, an automatic and target-free online boresight calibration method for LiDAR-camera systems. SE-Calib first extracts the semantic edge features from both point clouds and images simultaneously using the 3-D semantic segmentation (3-D-SS) and 2-D semantic edge detection (2-D-SED) methods. The boresight parameters are then optimized with an adaptive solver and maximizing the soft semantic response consistency metric (SSRCM) scores iteratively. The SSRCM is designed to evaluate the coherence of cross-modular semantic edge features, and a confidence function is proposed to filter out unreliable optimization results. Experiments conducted on challenging urban datasets show an average boresight error of 0.206° (2.47 pixels in reprojection error), demonstrating the effectiveness and robustness of the proposed method. |
| first_indexed | 2024-10-01T02:40:25Z |
| format | Journal Article |
| id | ntu-10356/169994 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:40:25Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1699942023-08-21T02:18:14Z SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes Liao, Youqi Li, Jianping Kang, Shuhao Li, Qiang Zhu, Guifang Yuan, Shenghai Dong, Zhen Yang, Bisheng School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Boresight Parameters Mobile Mapping System Rigorous boresight calibration between light detection and ranging (LiDAR) and the camera is crucial for geometry and optical information fusion in Earth observation and robotic applications. Although boresight parameters can be obtained through precalibration with artificial targets, unforeseen movement of sensors during data collection can lead to significant errors in the boresight parameters. To address this issue, we propose SE-Calib, an automatic and target-free online boresight calibration method for LiDAR-camera systems. SE-Calib first extracts the semantic edge features from both point clouds and images simultaneously using the 3-D semantic segmentation (3-D-SS) and 2-D semantic edge detection (2-D-SED) methods. The boresight parameters are then optimized with an adaptive solver and maximizing the soft semantic response consistency metric (SSRCM) scores iteratively. The SSRCM is designed to evaluate the coherence of cross-modular semantic edge features, and a confidence function is proposed to filter out unreliable optimization results. Experiments conducted on challenging urban datasets show an average boresight error of 0.206° (2.47 pixels in reprojection error), demonstrating the effectiveness and robustness of the proposed method. This work was supported in part by the National Natural Science Foundation Project under Grant 42130105 and Grant 42201477; in part by the China Postdoctoral Science Foundation under Grant 2022M712441 and Grant 2022TQ0234; and in part by the Open Fund of Key Laboratory of Urban Spatial Information, Ministry of Natural Resources, under Grant 2023ZD001. 2023-08-21T02:18:14Z 2023-08-21T02:18:14Z 2023 Journal Article Liao, Y., Li, J., Kang, S., Li, Q., Zhu, G., Yuan, S., Dong, Z. & Yang, B. (2023). SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes. IEEE Transactions On Geoscience and Remote Sensing, 61, 1000513-. https://dx.doi.org/10.1109/TGRS.2023.3278024 0196-2892 https://hdl.handle.net/10356/169994 10.1109/TGRS.2023.3278024 2-s2.0-85160227572 61 1000513 en IEEE Transactions on Geoscience and Remote Sensing © 2023 IEEE. All rights reserved. |
| spellingShingle | Engineering::Electrical and electronic engineering Boresight Parameters Mobile Mapping System Liao, Youqi Li, Jianping Kang, Shuhao Li, Qiang Zhu, Guifang Yuan, Shenghai Dong, Zhen Yang, Bisheng SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes |
| title | SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes |
| title_full | SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes |
| title_fullStr | SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes |
| title_full_unstemmed | SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes |
| title_short | SE-Calib: semantic edge-based LiDAR-camera boresight online calibration in urban scenes |
| title_sort | se calib semantic edge based lidar camera boresight online calibration in urban scenes |
| topic | Engineering::Electrical and electronic engineering Boresight Parameters Mobile Mapping System |
| url | https://hdl.handle.net/10356/169994 |
| work_keys_str_mv | AT liaoyouqi secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT lijianping secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT kangshuhao secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT liqiang secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT zhuguifang secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT yuanshenghai secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT dongzhen secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes AT yangbisheng secalibsemanticedgebasedlidarcameraboresightonlinecalibrationinurbanscenes |