Deep Photometric Stereo Network with Multi-Scale Feature Aggregation
We present photometric stereo algorithms robust to non-Lambertian reflection, which are based on a convolutional neural network in which surface normals of objects with complex geometry and surface reflectance are estimated from a given set of an arbitrary number of images. These images are taken fr...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/21/6261 |
| _version_ | 1797548963507732480 |
|---|---|
| author | Chanki Yu Sang Wook Lee |
| author_facet | Chanki Yu Sang Wook Lee |
| author_sort | Chanki Yu |
| collection | DOAJ |
| description | We present photometric stereo algorithms robust to non-Lambertian reflection, which are based on a convolutional neural network in which surface normals of objects with complex geometry and surface reflectance are estimated from a given set of an arbitrary number of images. These images are taken from the same viewpoint under different directional illumination conditions. The proposed method focuses on surface normal estimation, where multi-scale feature aggregation is proposed to obtain a more accurate surface normal, and max pooling is adopted to obtain an intermediate order-agnostic representation in the photometric stereo scenario. The proposed multi-scale feature aggregation scheme using feature concatenation is easily incorporated into existing photometric stereo network architectures. Our experiments were performed with a DiLiGent photometric stereo benchmark dataset consisting of ten real objects, and they demonstrated that the accuracies of our calibrated and uncalibrated photometric stereo approaches were improved over those of baseline methods. In particular, our experiments also demonstrated that our uncalibrated photometric stereo outperformed the state-of-the-art method. Our work is the first to consider the multi-scale feature aggregation in photometric stereo, and we showed that our proposed multi-scale fusion scheme estimated the surface normal accurately and was beneficial to improving performance. |
| first_indexed | 2024-03-10T15:08:13Z |
| format | Article |
| id | doaj.art-96a928a1ab02403ca8204f2272ded552 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T15:08:13Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-96a928a1ab02403ca8204f2272ded5522023-11-20T19:35:16ZengMDPI AGSensors1424-82202020-11-012021626110.3390/s20216261Deep Photometric Stereo Network with Multi-Scale Feature AggregationChanki Yu0Sang Wook Lee1Department of Media Technology, Graduate School of Media, Sogang University, Seoul 04107, KoreaDepartment of Media Technology, Graduate School of Media, Sogang University, Seoul 04107, KoreaWe present photometric stereo algorithms robust to non-Lambertian reflection, which are based on a convolutional neural network in which surface normals of objects with complex geometry and surface reflectance are estimated from a given set of an arbitrary number of images. These images are taken from the same viewpoint under different directional illumination conditions. The proposed method focuses on surface normal estimation, where multi-scale feature aggregation is proposed to obtain a more accurate surface normal, and max pooling is adopted to obtain an intermediate order-agnostic representation in the photometric stereo scenario. The proposed multi-scale feature aggregation scheme using feature concatenation is easily incorporated into existing photometric stereo network architectures. Our experiments were performed with a DiLiGent photometric stereo benchmark dataset consisting of ten real objects, and they demonstrated that the accuracies of our calibrated and uncalibrated photometric stereo approaches were improved over those of baseline methods. In particular, our experiments also demonstrated that our uncalibrated photometric stereo outperformed the state-of-the-art method. Our work is the first to consider the multi-scale feature aggregation in photometric stereo, and we showed that our proposed multi-scale fusion scheme estimated the surface normal accurately and was beneficial to improving performance.https://www.mdpi.com/1424-8220/20/21/6261deep learningcomputer visionconvolutional neural networkphotometric stereo |
| spellingShingle | Chanki Yu Sang Wook Lee Deep Photometric Stereo Network with Multi-Scale Feature Aggregation Sensors deep learning computer vision convolutional neural network photometric stereo |
| title | Deep Photometric Stereo Network with Multi-Scale Feature Aggregation |
| title_full | Deep Photometric Stereo Network with Multi-Scale Feature Aggregation |
| title_fullStr | Deep Photometric Stereo Network with Multi-Scale Feature Aggregation |
| title_full_unstemmed | Deep Photometric Stereo Network with Multi-Scale Feature Aggregation |
| title_short | Deep Photometric Stereo Network with Multi-Scale Feature Aggregation |
| title_sort | deep photometric stereo network with multi scale feature aggregation |
| topic | deep learning computer vision convolutional neural network photometric stereo |
| url | https://www.mdpi.com/1424-8220/20/21/6261 |
| work_keys_str_mv | AT chankiyu deepphotometricstereonetworkwithmultiscalefeatureaggregation AT sangwooklee deepphotometricstereonetworkwithmultiscalefeatureaggregation |