A Joint 2D-3D Complementary Network for Stereo Matching
Stereo matching is an important research field of computer vision. Due to the dimension of cost aggregation, current neural network-based stereo methods are difficult to trade-off speed and accuracy. To this end, we integrate fast 2D stereo methods with accurate 3D networks to improve performance an...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/4/1430 |
| _version_ | 1797396045753221120 |
|---|---|
| author | Xiaogang Jia Wei Chen Zhengfa Liang Xin Luo Mingfei Wu Chen Li Yulin He Yusong Tan Libo Huang |
| author_facet | Xiaogang Jia Wei Chen Zhengfa Liang Xin Luo Mingfei Wu Chen Li Yulin He Yusong Tan Libo Huang |
| author_sort | Xiaogang Jia |
| collection | DOAJ |
| description | Stereo matching is an important research field of computer vision. Due to the dimension of cost aggregation, current neural network-based stereo methods are difficult to trade-off speed and accuracy. To this end, we integrate fast 2D stereo methods with accurate 3D networks to improve performance and reduce running time. We leverage a 2D encoder-decoder network to generate a rough disparity map and construct a disparity range to guide the 3D aggregation network, which can significantly improve the accuracy and reduce the computational cost. We use a stacked hourglass structure to refine the disparity from coarse to fine. We evaluated our method on three public datasets. According to the KITTI official website results, Our network can generate an accurate result in 80 ms on a modern GPU. Compared to other 2D stereo networks (AANet, DeepPruner, FADNet, etc.), our network has a big improvement in accuracy. Meanwhile, it is significantly faster than other 3D stereo networks (5× than PSMNet, 7.5× than CSN and 22.5× than GANet, etc.), demonstrating the effectiveness of our method. |
| first_indexed | 2024-03-09T00:45:38Z |
| format | Article |
| id | doaj.art-630615f6fb164d7bbf21b2886ee25980 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T00:45:38Z |
| publishDate | 2021-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-630615f6fb164d7bbf21b2886ee259802023-12-11T17:32:52ZengMDPI AGSensors1424-82202021-02-01214143010.3390/s21041430A Joint 2D-3D Complementary Network for Stereo MatchingXiaogang Jia0Wei Chen1Zhengfa Liang2Xin Luo3Mingfei Wu4Chen Li5Yulin He6Yusong Tan7Libo Huang8College of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaStereo matching is an important research field of computer vision. Due to the dimension of cost aggregation, current neural network-based stereo methods are difficult to trade-off speed and accuracy. To this end, we integrate fast 2D stereo methods with accurate 3D networks to improve performance and reduce running time. We leverage a 2D encoder-decoder network to generate a rough disparity map and construct a disparity range to guide the 3D aggregation network, which can significantly improve the accuracy and reduce the computational cost. We use a stacked hourglass structure to refine the disparity from coarse to fine. We evaluated our method on three public datasets. According to the KITTI official website results, Our network can generate an accurate result in 80 ms on a modern GPU. Compared to other 2D stereo networks (AANet, DeepPruner, FADNet, etc.), our network has a big improvement in accuracy. Meanwhile, it is significantly faster than other 3D stereo networks (5× than PSMNet, 7.5× than CSN and 22.5× than GANet, etc.), demonstrating the effectiveness of our method.https://www.mdpi.com/1424-8220/21/4/1430stereo matchingdepth estimationcomputer vision |
| spellingShingle | Xiaogang Jia Wei Chen Zhengfa Liang Xin Luo Mingfei Wu Chen Li Yulin He Yusong Tan Libo Huang A Joint 2D-3D Complementary Network for Stereo Matching Sensors stereo matching depth estimation computer vision |
| title | A Joint 2D-3D Complementary Network for Stereo Matching |
| title_full | A Joint 2D-3D Complementary Network for Stereo Matching |
| title_fullStr | A Joint 2D-3D Complementary Network for Stereo Matching |
| title_full_unstemmed | A Joint 2D-3D Complementary Network for Stereo Matching |
| title_short | A Joint 2D-3D Complementary Network for Stereo Matching |
| title_sort | joint 2d 3d complementary network for stereo matching |
| topic | stereo matching depth estimation computer vision |
| url | https://www.mdpi.com/1424-8220/21/4/1430 |
| work_keys_str_mv | AT xiaogangjia ajoint2d3dcomplementarynetworkforstereomatching AT weichen ajoint2d3dcomplementarynetworkforstereomatching AT zhengfaliang ajoint2d3dcomplementarynetworkforstereomatching AT xinluo ajoint2d3dcomplementarynetworkforstereomatching AT mingfeiwu ajoint2d3dcomplementarynetworkforstereomatching AT chenli ajoint2d3dcomplementarynetworkforstereomatching AT yulinhe ajoint2d3dcomplementarynetworkforstereomatching AT yusongtan ajoint2d3dcomplementarynetworkforstereomatching AT libohuang ajoint2d3dcomplementarynetworkforstereomatching AT xiaogangjia joint2d3dcomplementarynetworkforstereomatching AT weichen joint2d3dcomplementarynetworkforstereomatching AT zhengfaliang joint2d3dcomplementarynetworkforstereomatching AT xinluo joint2d3dcomplementarynetworkforstereomatching AT mingfeiwu joint2d3dcomplementarynetworkforstereomatching AT chenli joint2d3dcomplementarynetworkforstereomatching AT yulinhe joint2d3dcomplementarynetworkforstereomatching AT yusongtan joint2d3dcomplementarynetworkforstereomatching AT libohuang joint2d3dcomplementarynetworkforstereomatching |