A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes
Textureless building surfaces composed of homogenized pixels could lead to failure of photometric consistency. However, textureless regions widely present in artificial scenes usually exhibit strong planarity enabling depth estimation of textureless regions with planar priors. However, existing meth...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10018887/ |
| _version_ | 1797846515306201088 |
|---|---|
| author | Nan Huang Zijie Huang Chuanyu Fu Hewen Zhou Yimin Xia Weijun Li Xiaoming Xiong Shuting Cai |
| author_facet | Nan Huang Zijie Huang Chuanyu Fu Hewen Zhou Yimin Xia Weijun Li Xiaoming Xiong Shuting Cai |
| author_sort | Nan Huang |
| collection | DOAJ |
| description | Textureless building surfaces composed of homogenized pixels could lead to failure of photometric consistency. However, textureless regions widely present in artificial scenes usually exhibit strong planarity enabling depth estimation of textureless regions with planar priors. However, existing methods for generating planar priors suffer from oversegmentation of large planes with textureless regions, which indicates that planarity is not fully exploited. In this study, we propose a novel generation method of planar prior by combining mean-shift clustering and superpixel segmentation. The planarity is fully utilized given preferential generation of planar priors for large planes with textureless regions in artificial scenes. Finally, a probabilistic graphical model is used to adopt the planar priors and smoothing constraints into depth estimation process. The image gradient is used as a criterion of the degree of texture to adaptively adjust the weights of different constraints. Experimental results on the benchmark dataset ETH3D, UDD5, and SenseFly demonstrate that the proposed method can effectively recover the depth information of textureless regions in high-resolution images to obtain highly complete three-dimensional (3-D) models of artificial scenes. |
| first_indexed | 2024-04-09T17:57:20Z |
| format | Article |
| id | doaj.art-d65bdef4ec5b45cba28fbab7c901a22b |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-04-09T17:57:20Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-d65bdef4ec5b45cba28fbab7c901a22b2023-04-14T23:00:11ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01163676369610.1109/JSTARS.2023.323758810018887A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial ScenesNan Huang0https://orcid.org/0000-0002-5246-6369Zijie Huang1https://orcid.org/0000-0002-9577-3844Chuanyu Fu2Hewen Zhou3Yimin Xia4Weijun Li5https://orcid.org/0000-0002-1227-0791Xiaoming Xiong6https://orcid.org/0000-0002-2421-7621Shuting Cai7https://orcid.org/0000-0002-2842-6439School of Automation, Guangdong University of Technology, Guangzhou, ChinaSchool of Automation, Guangdong University of Technology, Guangzhou, ChinaSchool of Automation, Guangdong University of Technology, Guangzhou, ChinaZhuhai Amicro Semiconductor Limited Company, Zhuhai, ChinaSchool of Automation, Guangdong University of Technology, Guangzhou, ChinaSchool of Automation, Guangdong University of Technology, Guangzhou, ChinaSchool of Automation, Guangdong University of Technology, Guangzhou, ChinaSchool of Automation, Guangdong University of Technology, Guangzhou, ChinaTextureless building surfaces composed of homogenized pixels could lead to failure of photometric consistency. However, textureless regions widely present in artificial scenes usually exhibit strong planarity enabling depth estimation of textureless regions with planar priors. However, existing methods for generating planar priors suffer from oversegmentation of large planes with textureless regions, which indicates that planarity is not fully exploited. In this study, we propose a novel generation method of planar prior by combining mean-shift clustering and superpixel segmentation. The planarity is fully utilized given preferential generation of planar priors for large planes with textureless regions in artificial scenes. Finally, a probabilistic graphical model is used to adopt the planar priors and smoothing constraints into depth estimation process. The image gradient is used as a criterion of the degree of texture to adaptively adjust the weights of different constraints. Experimental results on the benchmark dataset ETH3D, UDD5, and SenseFly demonstrate that the proposed method can effectively recover the depth information of textureless regions in high-resolution images to obtain highly complete three-dimensional (3-D) models of artificial scenes.https://ieeexplore.ieee.org/document/10018887/Planar priorsimage segmentationmultiview stereo (MVS)point cloudsremote sensingthree-dimensional (3-D) building reconstruction |
| spellingShingle | Nan Huang Zijie Huang Chuanyu Fu Hewen Zhou Yimin Xia Weijun Li Xiaoming Xiong Shuting Cai A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Planar priors image segmentation multiview stereo (MVS) point clouds remote sensing three-dimensional (3-D) building reconstruction |
| title | A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes |
| title_full | A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes |
| title_fullStr | A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes |
| title_full_unstemmed | A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes |
| title_short | A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes |
| title_sort | multiview stereo algorithm based on image segmentation guided generation of planar prior for textureless regions of artificial scenes |
| topic | Planar priors image segmentation multiview stereo (MVS) point clouds remote sensing three-dimensional (3-D) building reconstruction |
| url | https://ieeexplore.ieee.org/document/10018887/ |
| work_keys_str_mv | AT nanhuang amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT zijiehuang amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT chuanyufu amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT hewenzhou amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT yiminxia amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT weijunli amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT xiaomingxiong amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT shutingcai amultiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT nanhuang multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT zijiehuang multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT chuanyufu multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT hewenzhou multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT yiminxia multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT weijunli multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT xiaomingxiong multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes AT shutingcai multiviewstereoalgorithmbasedonimagesegmentationguidedgenerationofplanarpriorfortexturelessregionsofartificialscenes |