Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps
Depth estimation is a classical problem in computer vision, which typically relies on either a depth sensor or stereo matching alone. The depth sensor provides real-time estimates in repetitive and textureless regions where stereo matching is not effective. However, stereo matching can obtain more a...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2015-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1424-8220/15/8/20894 |
| _version_ | 1811185063569653760 |
|---|---|
| author | Jing Liu Chunpeng Li Xuefeng Fan Zhaoqi Wang |
| author_facet | Jing Liu Chunpeng Li Xuefeng Fan Zhaoqi Wang |
| author_sort | Jing Liu |
| collection | DOAJ |
| description | Depth estimation is a classical problem in computer vision, which typically relies on either a depth sensor or stereo matching alone. The depth sensor provides real-time estimates in repetitive and textureless regions where stereo matching is not effective. However, stereo matching can obtain more accurate results in rich texture regions and object boundaries where the depth sensor often fails. We fuse stereo matching and the depth sensor using their complementary characteristics to improve the depth estimation. Here, texture information is incorporated as a constraint to restrict the pixel’s scope of potential disparities and to reduce noise in repetitive and textureless regions. Furthermore, a novel pseudo-two-layer model is used to represent the relationship between disparities in different pixels and segments. It is more robust to luminance variation by treating information obtained from a depth sensor as prior knowledge. Segmentation is viewed as a soft constraint to reduce ambiguities caused by under- or over-segmentation. Compared to the average error rate 3.27% of the previous state-of-the-art methods, our method provides an average error rate of 2.61% on the Middlebury datasets, which shows that our method performs almost 20% better than other “fused” algorithms in the aspect of precision. |
| first_indexed | 2024-04-11T13:22:58Z |
| format | Article |
| id | doaj.art-f3834d6e9311471ab390e255b32a4b25 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T13:22:58Z |
| publishDate | 2015-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-f3834d6e9311471ab390e255b32a4b252022-12-22T04:22:09ZengMDPI AGSensors1424-82202015-08-01158208942092410.3390/s150820894s150820894Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth MapsJing Liu0Chunpeng Li1Xuefeng Fan2Zhaoqi Wang3The Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute ofComputing Technology, Chinese Academy of Sciences, No.6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190, ChinaThe Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute ofComputing Technology, Chinese Academy of Sciences, No.6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190, ChinaThe Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute ofComputing Technology, Chinese Academy of Sciences, No.6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190, ChinaThe Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute ofComputing Technology, Chinese Academy of Sciences, No.6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190, ChinaDepth estimation is a classical problem in computer vision, which typically relies on either a depth sensor or stereo matching alone. The depth sensor provides real-time estimates in repetitive and textureless regions where stereo matching is not effective. However, stereo matching can obtain more accurate results in rich texture regions and object boundaries where the depth sensor often fails. We fuse stereo matching and the depth sensor using their complementary characteristics to improve the depth estimation. Here, texture information is incorporated as a constraint to restrict the pixel’s scope of potential disparities and to reduce noise in repetitive and textureless regions. Furthermore, a novel pseudo-two-layer model is used to represent the relationship between disparities in different pixels and segments. It is more robust to luminance variation by treating information obtained from a depth sensor as prior knowledge. Segmentation is viewed as a soft constraint to reduce ambiguities caused by under- or over-segmentation. Compared to the average error rate 3.27% of the previous state-of-the-art methods, our method provides an average error rate of 2.61% on the Middlebury datasets, which shows that our method performs almost 20% better than other “fused” algorithms in the aspect of precision.http://www.mdpi.com/1424-8220/15/8/20894stereo matchingdepth sensormultiscale pseudo-two-layer modelsegmentationtexture constraintfusion move |
| spellingShingle | Jing Liu Chunpeng Li Xuefeng Fan Zhaoqi Wang Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps Sensors stereo matching depth sensor multiscale pseudo-two-layer model segmentation texture constraint fusion move |
| title | Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps |
| title_full | Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps |
| title_fullStr | Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps |
| title_full_unstemmed | Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps |
| title_short | Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps |
| title_sort | reliable fusion of stereo matching and depth sensor for high quality dense depth maps |
| topic | stereo matching depth sensor multiscale pseudo-two-layer model segmentation texture constraint fusion move |
| url | http://www.mdpi.com/1424-8220/15/8/20894 |
| work_keys_str_mv | AT jingliu reliablefusionofstereomatchinganddepthsensorforhighqualitydensedepthmaps AT chunpengli reliablefusionofstereomatchinganddepthsensorforhighqualitydensedepthmaps AT xuefengfan reliablefusionofstereomatchinganddepthsensorforhighqualitydensedepthmaps AT zhaoqiwang reliablefusionofstereomatchinganddepthsensorforhighqualitydensedepthmaps |