Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo
Multi-spectral photometric stereo can recover pixel-wise surface normal from a single RGB image. The difficulty lies in that the intensity in each channel is the tangle of illumination, albedo and camera response; thus, an initial estimate of the normal is required in optimization-based solutions. I...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2018-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1424-8220/18/3/764 |
| _version_ | 1828151535361589248 |
|---|---|
| author | Liang Lu Lin Qi Yisong Luo Hengchao Jiao Junyu Dong |
| author_facet | Liang Lu Lin Qi Yisong Luo Hengchao Jiao Junyu Dong |
| author_sort | Liang Lu |
| collection | DOAJ |
| description | Multi-spectral photometric stereo can recover pixel-wise surface normal from a single RGB image. The difficulty lies in that the intensity in each channel is the tangle of illumination, albedo and camera response; thus, an initial estimate of the normal is required in optimization-based solutions. In this paper, we propose to make a rough depth estimation using the deep convolutional neural network (CNN) instead of using depth sensors or binocular stereo devices. Since high-resolution ground-truth data is expensive to obtain, we designed a network and trained it with rendered images of synthetic 3D objects. We use the model to predict initial normal of real-world objects and iteratively optimize the fine-scale geometry in the multi-spectral photometric stereo framework. The experimental results illustrate the improvement of the proposed method compared with existing methods. |
| first_indexed | 2024-04-11T21:59:08Z |
| format | Article |
| id | doaj.art-090db0c03caa42d88f2c1f27ce854c01 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T21:59:08Z |
| publishDate | 2018-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-090db0c03caa42d88f2c1f27ce854c012022-12-22T04:01:00ZengMDPI AGSensors1424-82202018-03-0118376410.3390/s18030764s18030764Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric StereoLiang Lu0Lin Qi1Yisong Luo2Hengchao Jiao3Junyu Dong4College of Information Science and Engineering, Ocean University of China, Qingdao 266100, ChinaCollege of Information Science and Engineering, Ocean University of China, Qingdao 266100, ChinaCollege of Information Science and Engineering, Ocean University of China, Qingdao 266100, ChinaCollege of Information Science and Engineering, Ocean University of China, Qingdao 266100, ChinaCollege of Information Science and Engineering, Ocean University of China, Qingdao 266100, ChinaMulti-spectral photometric stereo can recover pixel-wise surface normal from a single RGB image. The difficulty lies in that the intensity in each channel is the tangle of illumination, albedo and camera response; thus, an initial estimate of the normal is required in optimization-based solutions. In this paper, we propose to make a rough depth estimation using the deep convolutional neural network (CNN) instead of using depth sensors or binocular stereo devices. Since high-resolution ground-truth data is expensive to obtain, we designed a network and trained it with rendered images of synthetic 3D objects. We use the model to predict initial normal of real-world objects and iteratively optimize the fine-scale geometry in the multi-spectral photometric stereo framework. The experimental results illustrate the improvement of the proposed method compared with existing methods.http://www.mdpi.com/1424-8220/18/3/764depth estimationconvolutional neural networkmulti-spectral photometric stereo |
| spellingShingle | Liang Lu Lin Qi Yisong Luo Hengchao Jiao Junyu Dong Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo Sensors depth estimation convolutional neural network multi-spectral photometric stereo |
| title | Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo |
| title_full | Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo |
| title_fullStr | Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo |
| title_full_unstemmed | Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo |
| title_short | Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo |
| title_sort | three dimensional reconstruction from single image base on combination of cnn and multi spectral photometric stereo |
| topic | depth estimation convolutional neural network multi-spectral photometric stereo |
| url | http://www.mdpi.com/1424-8220/18/3/764 |
| work_keys_str_mv | AT lianglu threedimensionalreconstructionfromsingleimagebaseoncombinationofcnnandmultispectralphotometricstereo AT linqi threedimensionalreconstructionfromsingleimagebaseoncombinationofcnnandmultispectralphotometricstereo AT yisongluo threedimensionalreconstructionfromsingleimagebaseoncombinationofcnnandmultispectralphotometricstereo AT hengchaojiao threedimensionalreconstructionfromsingleimagebaseoncombinationofcnnandmultispectralphotometricstereo AT junyudong threedimensionalreconstructionfromsingleimagebaseoncombinationofcnnandmultispectralphotometricstereo |