Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching
We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) they assume the given sparse depth map is acc...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9558759/ |
| _version_ | 1818726358009774080 |
|---|---|
| author | Yasuhiro Yao Ryoichi Ishikawa Shingo Ando Kana Kurata Naoki Ito Jun Shimamura Takeshi Oishi |
| author_facet | Yasuhiro Yao Ryoichi Ishikawa Shingo Ando Kana Kurata Naoki Ito Jun Shimamura Takeshi Oishi |
| author_sort | Yasuhiro Yao |
| collection | DOAJ |
| description | We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) they assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice; (ii) they have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods. |
| first_indexed | 2024-12-17T21:56:56Z |
| format | Article |
| id | doaj.art-3a394510d6794b93bfe14bbbba86ca61 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-17T21:56:56Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-3a394510d6794b93bfe14bbbba86ca612022-12-21T21:31:05ZengIEEEIEEE Access2169-35362021-01-01913667413668610.1109/ACCESS.2021.31177109558759Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo MatchingYasuhiro Yao0https://orcid.org/0000-0002-4195-8229Ryoichi Ishikawa1https://orcid.org/0000-0001-6904-3437Shingo Ando2Kana Kurata3Naoki Ito4Jun Shimamura5https://orcid.org/0000-0002-3424-253XTakeshi Oishi6https://orcid.org/0000-0002-2010-2608Institute of Industrial Science, The University of Tokyo, Tokyo, JapanInstitute of Industrial Science, The University of Tokyo, Tokyo, JapanNTT Human Informatics Laboratories, Yokosuka, JapanNTT Human Informatics Laboratories, Yokosuka, JapanNTT Human Informatics Laboratories, Yokosuka, JapanNTT Human Informatics Laboratories, Yokosuka, JapanInstitute of Industrial Science, The University of Tokyo, Tokyo, JapanWe propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) they assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice; (ii) they have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods.https://ieeexplore.ieee.org/document/9558759/Computer visiondepth completionLiDARsensor fusionstereo matching |
| spellingShingle | Yasuhiro Yao Ryoichi Ishikawa Shingo Ando Kana Kurata Naoki Ito Jun Shimamura Takeshi Oishi Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching IEEE Access Computer vision depth completion LiDAR sensor fusion stereo matching |
| title | Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching |
| title_full | Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching |
| title_fullStr | Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching |
| title_full_unstemmed | Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching |
| title_short | Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching |
| title_sort | non learning stereo aided depth completion under mis projection via selective stereo matching |
| topic | Computer vision depth completion LiDAR sensor fusion stereo matching |
| url | https://ieeexplore.ieee.org/document/9558759/ |
| work_keys_str_mv | AT yasuhiroyao nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching AT ryoichiishikawa nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching AT shingoando nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching AT kanakurata nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching AT naokiito nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching AT junshimamura nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching AT takeshioishi nonlearningstereoaideddepthcompletionundermisprojectionviaselectivestereomatching |