Semantic Segmentation Leveraging Simultaneous Depth Estimation
Semantic segmentation is one of the most widely studied problems in computer vision communities, which makes a great contribution to a variety of applications. A lot of learning-based approaches, such as Convolutional Neural Network (CNN), have made a vast contribution to this problem. While rich co...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/3/690 |
| _version_ | 1797409348945707008 |
|---|---|
| author | Wenbo Sun Zhi Gao Jinqiang Cui Bharath Ramesh Bin Zhang Ziyao Li |
| author_facet | Wenbo Sun Zhi Gao Jinqiang Cui Bharath Ramesh Bin Zhang Ziyao Li |
| author_sort | Wenbo Sun |
| collection | DOAJ |
| description | Semantic segmentation is one of the most widely studied problems in computer vision communities, which makes a great contribution to a variety of applications. A lot of learning-based approaches, such as Convolutional Neural Network (CNN), have made a vast contribution to this problem. While rich context information of the input images can be learned from multi-scale receptive fields by convolutions with deep layers, traditional CNNs have great difficulty in learning the geometrical relationship and distribution of objects in the RGB image due to the lack of depth information, which may lead to an inferior segmentation quality. To solve this problem, we propose a method that improves segmentation quality with depth estimation on RGB images. Specifically, we estimate depth information on RGB images via a depth estimation network, and then feed the depth map into the CNN which is able to guide the semantic segmentation. Furthermore, in order to parse the depth map and RGB images simultaneously, we construct a multi-branch encoder–decoder network and fuse the RGB and depth features step by step. Extensive experimental evaluation on four baseline networks demonstrates that our proposed method can enhance the segmentation quality considerably and obtain better performance compared to other segmentation networks. |
| first_indexed | 2024-03-09T04:13:08Z |
| format | Article |
| id | doaj.art-a01b5c4311114cb4ba63fa971196fbd5 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T04:13:08Z |
| publishDate | 2021-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-a01b5c4311114cb4ba63fa971196fbd52023-12-03T13:58:25ZengMDPI AGSensors1424-82202021-01-0121369010.3390/s21030690Semantic Segmentation Leveraging Simultaneous Depth EstimationWenbo Sun0Zhi Gao1Jinqiang Cui2Bharath Ramesh3Bin Zhang4Ziyao Li5School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaPeng Cheng Laboratory, Shenzhen 518055, ChinaThe N.1 Institute for Health, National University of Singapore, Singapore 117411, SingaporeSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaSemantic segmentation is one of the most widely studied problems in computer vision communities, which makes a great contribution to a variety of applications. A lot of learning-based approaches, such as Convolutional Neural Network (CNN), have made a vast contribution to this problem. While rich context information of the input images can be learned from multi-scale receptive fields by convolutions with deep layers, traditional CNNs have great difficulty in learning the geometrical relationship and distribution of objects in the RGB image due to the lack of depth information, which may lead to an inferior segmentation quality. To solve this problem, we propose a method that improves segmentation quality with depth estimation on RGB images. Specifically, we estimate depth information on RGB images via a depth estimation network, and then feed the depth map into the CNN which is able to guide the semantic segmentation. Furthermore, in order to parse the depth map and RGB images simultaneously, we construct a multi-branch encoder–decoder network and fuse the RGB and depth features step by step. Extensive experimental evaluation on four baseline networks demonstrates that our proposed method can enhance the segmentation quality considerably and obtain better performance compared to other segmentation networks.https://www.mdpi.com/1424-8220/21/3/690CNNsemantic segmentationdepth estimationmulti-source feature fusion |
| spellingShingle | Wenbo Sun Zhi Gao Jinqiang Cui Bharath Ramesh Bin Zhang Ziyao Li Semantic Segmentation Leveraging Simultaneous Depth Estimation Sensors CNN semantic segmentation depth estimation multi-source feature fusion |
| title | Semantic Segmentation Leveraging Simultaneous Depth Estimation |
| title_full | Semantic Segmentation Leveraging Simultaneous Depth Estimation |
| title_fullStr | Semantic Segmentation Leveraging Simultaneous Depth Estimation |
| title_full_unstemmed | Semantic Segmentation Leveraging Simultaneous Depth Estimation |
| title_short | Semantic Segmentation Leveraging Simultaneous Depth Estimation |
| title_sort | semantic segmentation leveraging simultaneous depth estimation |
| topic | CNN semantic segmentation depth estimation multi-source feature fusion |
| url | https://www.mdpi.com/1424-8220/21/3/690 |
| work_keys_str_mv | AT wenbosun semanticsegmentationleveragingsimultaneousdepthestimation AT zhigao semanticsegmentationleveragingsimultaneousdepthestimation AT jinqiangcui semanticsegmentationleveragingsimultaneousdepthestimation AT bharathramesh semanticsegmentationleveragingsimultaneousdepthestimation AT binzhang semanticsegmentationleveragingsimultaneousdepthestimation AT ziyaoli semanticsegmentationleveragingsimultaneousdepthestimation |