High-Fidelity Depth Upsampling Using the Self-Learning Framework
This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning fr...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Multidisciplinary Digital Publishing Institute
2019
|
| Online Access: | http://hdl.handle.net/1721.1/120468 https://orcid.org/0000-0003-0468-1571 |
| _version_ | 1826197718046867456 |
|---|---|
| author | Shim, Inwook Kweon, In So Oh, Taehyun |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Shim, Inwook Kweon, In So Oh, Taehyun |
| author_sort | Shim, Inwook |
| collection | MIT |
| description | This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points. Keywords: depth upsampling; depth filtering; LiDAR; self-learning; self-supervised learning |
| first_indexed | 2024-09-23T10:52:04Z |
| format | Article |
| id | mit-1721.1/120468 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T10:52:04Z |
| publishDate | 2019 |
| publisher | Multidisciplinary Digital Publishing Institute |
| record_format | dspace |
| spelling | mit-1721.1/1204682022-09-27T15:34:49Z High-Fidelity Depth Upsampling Using the Self-Learning Framework Shim, Inwook Kweon, In So Oh, Taehyun Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Oh, Taehyun This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points. Keywords: depth upsampling; depth filtering; LiDAR; self-learning; self-supervised learning 2019-02-15T20:07:02Z 2019-02-15T20:07:02Z 2018-12 2018-10 2019-01-24T09:22:08Z Article http://purl.org/eprint/type/JournalArticle 1424-8220 http://hdl.handle.net/1721.1/120468 Shim, Inwook et al. "High-Fidelity Depth Upsampling Using the Self-Learning Framework." Sensors 19, 1 (December 2018): 81 © The Authors https://orcid.org/0000-0003-0468-1571 http://dx.doi.org/10.3390/s19010081 Sensors Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ application/pdf Multidisciplinary Digital Publishing Institute Multidisciplinary Digital Publishing Institute |
| spellingShingle | Shim, Inwook Kweon, In So Oh, Taehyun High-Fidelity Depth Upsampling Using the Self-Learning Framework |
| title | High-Fidelity Depth Upsampling Using the Self-Learning Framework |
| title_full | High-Fidelity Depth Upsampling Using the Self-Learning Framework |
| title_fullStr | High-Fidelity Depth Upsampling Using the Self-Learning Framework |
| title_full_unstemmed | High-Fidelity Depth Upsampling Using the Self-Learning Framework |
| title_short | High-Fidelity Depth Upsampling Using the Self-Learning Framework |
| title_sort | high fidelity depth upsampling using the self learning framework |
| url | http://hdl.handle.net/1721.1/120468 https://orcid.org/0000-0003-0468-1571 |
| work_keys_str_mv | AT shiminwook highfidelitydepthupsamplingusingtheselflearningframework AT kweoninso highfidelitydepthupsamplingusingtheselflearningframework AT ohtaehyun highfidelitydepthupsamplingusingtheselflearningframework |