SPNet: Structure preserving network for depth completion
Depth completion aims to predict a dense depth map from a sparse one. Benefiting from the powerful ability of convolutional neural networks, recent depth completion methods have achieved remarkable performance. However, it is still a challenging problem to well preserve accurate depth structures, su...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9873174/?tool=EBI |
| _version_ | 1797942896275488768 |
|---|---|
| author | Tao Li Songning Luo Zhiwei Fan Qunbing Zhou Ting Hu |
| author_facet | Tao Li Songning Luo Zhiwei Fan Qunbing Zhou Ting Hu |
| author_sort | Tao Li |
| collection | DOAJ |
| description | Depth completion aims to predict a dense depth map from a sparse one. Benefiting from the powerful ability of convolutional neural networks, recent depth completion methods have achieved remarkable performance. However, it is still a challenging problem to well preserve accurate depth structures, such as tiny structures and object boundaries. To tackle this problem, we propose a structure preserving network (SPNet) in this paper. Firstly, an efficient multi-scale gradient extractor (MSGE) is proposed to extract useful multi-scale gradient images, which contain rich structural information that is helpful in recovering accurate depth. The MSGE is constructed based on the proposed semi-fixed depthwise separable convolution. Meanwhile, we adopt a stable gradient MAE loss (LGMAE) to provide additional depth gradient constrain for better structure reconstruction. Moreover, a multi-level feature fusion module (MFFM) is proposed to adaptively fuse the spatial details from low-level encoder and the semantic information from high-level decoder, which will incorporate more structural details into the depth modality. As demonstrated by experiments on NYUv2 and KITTI datasets, our method outperforms some state-of-the-art methods in terms of both quantitative and quantitative evaluations. |
| first_indexed | 2024-04-10T20:15:54Z |
| format | Article |
| id | doaj.art-af348eab6c114c33b763dc7f89686efb |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-04-10T20:15:54Z |
| publishDate | 2023-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-af348eab6c114c33b763dc7f89686efb2023-01-26T05:32:29ZengPublic Library of Science (PLoS)PLoS ONE1932-62032023-01-01181SPNet: Structure preserving network for depth completionTao LiSongning LuoZhiwei FanQunbing ZhouTing HuDepth completion aims to predict a dense depth map from a sparse one. Benefiting from the powerful ability of convolutional neural networks, recent depth completion methods have achieved remarkable performance. However, it is still a challenging problem to well preserve accurate depth structures, such as tiny structures and object boundaries. To tackle this problem, we propose a structure preserving network (SPNet) in this paper. Firstly, an efficient multi-scale gradient extractor (MSGE) is proposed to extract useful multi-scale gradient images, which contain rich structural information that is helpful in recovering accurate depth. The MSGE is constructed based on the proposed semi-fixed depthwise separable convolution. Meanwhile, we adopt a stable gradient MAE loss (LGMAE) to provide additional depth gradient constrain for better structure reconstruction. Moreover, a multi-level feature fusion module (MFFM) is proposed to adaptively fuse the spatial details from low-level encoder and the semantic information from high-level decoder, which will incorporate more structural details into the depth modality. As demonstrated by experiments on NYUv2 and KITTI datasets, our method outperforms some state-of-the-art methods in terms of both quantitative and quantitative evaluations.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9873174/?tool=EBI |
| spellingShingle | Tao Li Songning Luo Zhiwei Fan Qunbing Zhou Ting Hu SPNet: Structure preserving network for depth completion PLoS ONE |
| title | SPNet: Structure preserving network for depth completion |
| title_full | SPNet: Structure preserving network for depth completion |
| title_fullStr | SPNet: Structure preserving network for depth completion |
| title_full_unstemmed | SPNet: Structure preserving network for depth completion |
| title_short | SPNet: Structure preserving network for depth completion |
| title_sort | spnet structure preserving network for depth completion |
| url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9873174/?tool=EBI |
| work_keys_str_mv | AT taoli spnetstructurepreservingnetworkfordepthcompletion AT songningluo spnetstructurepreservingnetworkfordepthcompletion AT zhiweifan spnetstructurepreservingnetworkfordepthcompletion AT qunbingzhou spnetstructurepreservingnetworkfordepthcompletion AT tinghu spnetstructurepreservingnetworkfordepthcompletion |