An Adaptive Refinement Scheme for Depth Estimation Networks
Deep learning has proved to be a breakthrough in depth generation. However, the generalization ability of deep networks is still limited, and they cannot maintain a satisfactory performance on some inputs. By addressing a similar problem in the segmentation field, a feature backpropagating refinemen...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/24/9755 |
| _version_ | 1797455302881181696 |
|---|---|
| author | Amin Alizadeh Naeini Mohammad Moein Sheikholeslami Gunho Sohn |
| author_facet | Amin Alizadeh Naeini Mohammad Moein Sheikholeslami Gunho Sohn |
| author_sort | Amin Alizadeh Naeini |
| collection | DOAJ |
| description | Deep learning has proved to be a breakthrough in depth generation. However, the generalization ability of deep networks is still limited, and they cannot maintain a satisfactory performance on some inputs. By addressing a similar problem in the segmentation field, a feature backpropagating refinement scheme (f-BRS) has been proposed to refine predictions in the inference time. f-BRS adapts an intermediate activation function to each input by using user clicks as sparse labels. Given the similarity between user clicks and sparse depth maps, this paper aims to extend the application of f-BRS to depth prediction. Our experiments show that f-BRS, fused with a depth estimation baseline, is trapped in local optima, and fails to improve the network predictions. To resolve that, we propose a double-stage adaptive refinement scheme (DARS). In the first stage, a Delaunay-based correction module significantly improves the depth generated by a baseline network. In the second stage, a particle swarm optimizer (PSO) delineates the estimation through fine-tuning f-BRS parameters—that is, scales and biases. DARS is evaluated on an outdoor benchmark, KITTI, and an indoor benchmark, NYUv2, while for both, the network is pre-trained on KITTI. The proposed scheme was effective on both datasets. |
| first_indexed | 2024-03-09T15:52:30Z |
| format | Article |
| id | doaj.art-53546baa4ed04cbdb18b149ebc463c08 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T15:52:30Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-53546baa4ed04cbdb18b149ebc463c082023-11-24T17:54:48ZengMDPI AGSensors1424-82202022-12-012224975510.3390/s22249755An Adaptive Refinement Scheme for Depth Estimation NetworksAmin Alizadeh Naeini0Mohammad Moein Sheikholeslami1Gunho Sohn2Department of Earth and Space Science and Engineering, Lassonde School of Engineering, York University, 4700 Keele Street, Toronto, ON M3J1P3, CanadaDepartment of Earth and Space Science and Engineering, Lassonde School of Engineering, York University, 4700 Keele Street, Toronto, ON M3J1P3, CanadaDepartment of Earth and Space Science and Engineering, Lassonde School of Engineering, York University, 4700 Keele Street, Toronto, ON M3J1P3, CanadaDeep learning has proved to be a breakthrough in depth generation. However, the generalization ability of deep networks is still limited, and they cannot maintain a satisfactory performance on some inputs. By addressing a similar problem in the segmentation field, a feature backpropagating refinement scheme (f-BRS) has been proposed to refine predictions in the inference time. f-BRS adapts an intermediate activation function to each input by using user clicks as sparse labels. Given the similarity between user clicks and sparse depth maps, this paper aims to extend the application of f-BRS to depth prediction. Our experiments show that f-BRS, fused with a depth estimation baseline, is trapped in local optima, and fails to improve the network predictions. To resolve that, we propose a double-stage adaptive refinement scheme (DARS). In the first stage, a Delaunay-based correction module significantly improves the depth generated by a baseline network. In the second stage, a particle swarm optimizer (PSO) delineates the estimation through fine-tuning f-BRS parameters—that is, scales and biases. DARS is evaluated on an outdoor benchmark, KITTI, and an indoor benchmark, NYUv2, while for both, the network is pre-trained on KITTI. The proposed scheme was effective on both datasets.https://www.mdpi.com/1424-8220/22/24/9755depth estimationoptimizationdeep learning |
| spellingShingle | Amin Alizadeh Naeini Mohammad Moein Sheikholeslami Gunho Sohn An Adaptive Refinement Scheme for Depth Estimation Networks Sensors depth estimation optimization deep learning |
| title | An Adaptive Refinement Scheme for Depth Estimation Networks |
| title_full | An Adaptive Refinement Scheme for Depth Estimation Networks |
| title_fullStr | An Adaptive Refinement Scheme for Depth Estimation Networks |
| title_full_unstemmed | An Adaptive Refinement Scheme for Depth Estimation Networks |
| title_short | An Adaptive Refinement Scheme for Depth Estimation Networks |
| title_sort | adaptive refinement scheme for depth estimation networks |
| topic | depth estimation optimization deep learning |
| url | https://www.mdpi.com/1424-8220/22/24/9755 |
| work_keys_str_mv | AT aminalizadehnaeini anadaptiverefinementschemefordepthestimationnetworks AT mohammadmoeinsheikholeslami anadaptiverefinementschemefordepthestimationnetworks AT gunhosohn anadaptiverefinementschemefordepthestimationnetworks AT aminalizadehnaeini adaptiverefinementschemefordepthestimationnetworks AT mohammadmoeinsheikholeslami adaptiverefinementschemefordepthestimationnetworks AT gunhosohn adaptiverefinementschemefordepthestimationnetworks |