Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps
This study proposes a multiview stereo (MVS) method that is based on the forward and backward propagation of matching costs for the incremental refinement of multiview disparity maps. The volume-based MVS method requires numerous three-dimensional (3D) memory volumes to compute, store, and process t...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9994690/ |
| _version_ | 1797974442792452096 |
|---|---|
| author | Min-Jae Lee Soon-Yong Park |
| author_facet | Min-Jae Lee Soon-Yong Park |
| author_sort | Min-Jae Lee |
| collection | DOAJ |
| description | This study proposes a multiview stereo (MVS) method that is based on the forward and backward propagation of matching costs for the incremental refinement of multiview disparity maps. The volume-based MVS method requires numerous three-dimensional (3D) memory volumes to compute, store, and process the stereo matching costs. However, owing to memory limitations, conventional MVS methods allocate the memory of the 3D cost volumes only to the reference and its neighbor views. Thus, each reference view can only use the matching costs from a limited number of neighbor views. This study addresses this inherent MVS problem and proposes a new method by employing a forward and backward cost propagation (FBCP). First, a subpart of the input views is used to obtain disparity maps with a dense MVS method. Once all matching costs of the subpart views are sufficiently refined, the FBCP is performed for a new neighbor view. Immediately after the cost volume of the new view is computed, all matching costs of the subpart are forward propagated and fused with the initial cost of the new view. Furthermore, the new fused cost is backward propagated into the subpart to refine the previous costs again using the new fused cost. All cost volumes can be incrementally computed and refined without any limitation on the number of views using the proposed FBCP scheme. In the final step, all disparity maps are obtained from the refined cost volumes and fused into single point clouds. Moreover, we propose the use of surface consensus to obtain accurate fused point clouds for the fusion of the disparity maps. The performance of the proposed method is evaluated using the fused point clouds. The proposed method achieves less than 0.5 mm in mean distance error and about 82 percentage F-score within 2 mm distance threshold value. |
| first_indexed | 2024-04-11T04:20:02Z |
| format | Article |
| id | doaj.art-e48485df22e94da19783db1c91ba14c5 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T04:20:02Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e48485df22e94da19783db1c91ba14c52022-12-31T00:01:11ZengIEEEIEEE Access2169-35362022-01-011013407413408510.1109/ACCESS.2022.32309499994690Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity MapsMin-Jae Lee0https://orcid.org/0000-0003-0663-6109Soon-Yong Park1https://orcid.org/0000-0001-5090-9667School of Electronic and Electrical Engineering, Kyungpook National University, Daegu, South KoreaSchool of Electronic and Electrical Engineering, Kyungpook National University, Daegu, South KoreaThis study proposes a multiview stereo (MVS) method that is based on the forward and backward propagation of matching costs for the incremental refinement of multiview disparity maps. The volume-based MVS method requires numerous three-dimensional (3D) memory volumes to compute, store, and process the stereo matching costs. However, owing to memory limitations, conventional MVS methods allocate the memory of the 3D cost volumes only to the reference and its neighbor views. Thus, each reference view can only use the matching costs from a limited number of neighbor views. This study addresses this inherent MVS problem and proposes a new method by employing a forward and backward cost propagation (FBCP). First, a subpart of the input views is used to obtain disparity maps with a dense MVS method. Once all matching costs of the subpart views are sufficiently refined, the FBCP is performed for a new neighbor view. Immediately after the cost volume of the new view is computed, all matching costs of the subpart are forward propagated and fused with the initial cost of the new view. Furthermore, the new fused cost is backward propagated into the subpart to refine the previous costs again using the new fused cost. All cost volumes can be incrementally computed and refined without any limitation on the number of views using the proposed FBCP scheme. In the final step, all disparity maps are obtained from the refined cost volumes and fused into single point clouds. Moreover, we propose the use of surface consensus to obtain accurate fused point clouds for the fusion of the disparity maps. The performance of the proposed method is evaluated using the fused point clouds. The proposed method achieves less than 0.5 mm in mean distance error and about 82 percentage F-score within 2 mm distance threshold value.https://ieeexplore.ieee.org/document/9994690/Multiview stereo3D reconstructionmatching cost volumecost propagationdisparity map refinement |
| spellingShingle | Min-Jae Lee Soon-Yong Park Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps IEEE Access Multiview stereo 3D reconstruction matching cost volume cost propagation disparity map refinement |
| title | Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps |
| title_full | Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps |
| title_fullStr | Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps |
| title_full_unstemmed | Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps |
| title_short | Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps |
| title_sort | forward and backward propagation of stereo matching cost for incremental refinement of multiview disparity maps |
| topic | Multiview stereo 3D reconstruction matching cost volume cost propagation disparity map refinement |
| url | https://ieeexplore.ieee.org/document/9994690/ |
| work_keys_str_mv | AT minjaelee forwardandbackwardpropagationofstereomatchingcostforincrementalrefinementofmultiviewdisparitymaps AT soonyongpark forwardandbackwardpropagationofstereomatchingcostforincrementalrefinementofmultiviewdisparitymaps |