Single-Shot Three-Dimensional Measurement by Fringe Analysis Network
Fringe projection profilometry (FPP) has been broadly applied in three-dimensional (3D) measurements, but the existing multi-shot methods, which mostly utilize phase-shifting techniques, are heavily affected by the disturbance of vibration and cannot be used in dynamic scenes. In this work, a single...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Photonics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2304-6732/10/4/417 |
| _version_ | 1797603836364324864 |
|---|---|
| author | Mingzhu Wan Lingbao Kong Xing Peng |
| author_facet | Mingzhu Wan Lingbao Kong Xing Peng |
| author_sort | Mingzhu Wan |
| collection | DOAJ |
| description | Fringe projection profilometry (FPP) has been broadly applied in three-dimensional (3D) measurements, but the existing multi-shot methods, which mostly utilize phase-shifting techniques, are heavily affected by the disturbance of vibration and cannot be used in dynamic scenes. In this work, a single-shot 3D measurement method using a deep neural network named the Fringe Analysis Network (FrANet) is proposed. The FrANet is composed of a phase retrieval subnetwork, phase unwrapping subnetwork, and refinement subnetwork. The combination of multiple subnetworks can help to recover long-range information that is missing for a single U-Net. A two-stage training strategy in which the FrANet network is pre-trained using fringe pattern reprojection and fine-tuned using ground truth phase maps is designed. Such a training strategy lowers the number of ground truth phase maps in the data set, saves time during data collection, and maintains the accuracy of supervised methods in real-world setups. Experimental studies were carried out on a setup FPP system. In the test set, the mean absolute error (MAE) of the refined absolute phase maps was 0.0114 rad, and the root mean square error (RMSE) of the 3D reconstruction results was 0.67 mm. The accuracy of the proposed method in dynamic scenes was evaluated by measuring moving standard spheres. The measurement of the sphere diameter maintained a high accuracy of 84 μm at a speed of 0.759 m/s. Two-stage training only requires 8800 fringe images in data acquisition, while supervised methods require 96,000 fringe images for the same number of iterations. Ablation studies verified the effectiveness of two training stages and three subnetworks. The proposed method achieved accurate single-shot 3D measurements comparable to those obtained using supervised methods and has a high data efficiency. This enables the accurate 3D shape measurement of moving or vibrating objects in industrial manufacturing and allows for further exploration of network architecture and training strategy with few training samples for single-shot 3D measurement. |
| first_indexed | 2024-03-11T04:37:43Z |
| format | Article |
| id | doaj.art-d93cd9c0c9614b8da43a8fc08ea6ea5b |
| institution | Directory Open Access Journal |
| issn | 2304-6732 |
| language | English |
| last_indexed | 2024-03-11T04:37:43Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Photonics |
| spelling | doaj.art-d93cd9c0c9614b8da43a8fc08ea6ea5b2023-11-17T20:57:45ZengMDPI AGPhotonics2304-67322023-04-0110441710.3390/photonics10040417Single-Shot Three-Dimensional Measurement by Fringe Analysis NetworkMingzhu Wan0Lingbao Kong1Xing Peng2Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, School of Information Science and Technology, Fudan University, Shanghai 200438, ChinaShanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, School of Information Science and Technology, Fudan University, Shanghai 200438, ChinaCollege of Intelligent Science and Technology, National University of Defense Technology, Changsha 410073, ChinaFringe projection profilometry (FPP) has been broadly applied in three-dimensional (3D) measurements, but the existing multi-shot methods, which mostly utilize phase-shifting techniques, are heavily affected by the disturbance of vibration and cannot be used in dynamic scenes. In this work, a single-shot 3D measurement method using a deep neural network named the Fringe Analysis Network (FrANet) is proposed. The FrANet is composed of a phase retrieval subnetwork, phase unwrapping subnetwork, and refinement subnetwork. The combination of multiple subnetworks can help to recover long-range information that is missing for a single U-Net. A two-stage training strategy in which the FrANet network is pre-trained using fringe pattern reprojection and fine-tuned using ground truth phase maps is designed. Such a training strategy lowers the number of ground truth phase maps in the data set, saves time during data collection, and maintains the accuracy of supervised methods in real-world setups. Experimental studies were carried out on a setup FPP system. In the test set, the mean absolute error (MAE) of the refined absolute phase maps was 0.0114 rad, and the root mean square error (RMSE) of the 3D reconstruction results was 0.67 mm. The accuracy of the proposed method in dynamic scenes was evaluated by measuring moving standard spheres. The measurement of the sphere diameter maintained a high accuracy of 84 μm at a speed of 0.759 m/s. Two-stage training only requires 8800 fringe images in data acquisition, while supervised methods require 96,000 fringe images for the same number of iterations. Ablation studies verified the effectiveness of two training stages and three subnetworks. The proposed method achieved accurate single-shot 3D measurements comparable to those obtained using supervised methods and has a high data efficiency. This enables the accurate 3D shape measurement of moving or vibrating objects in industrial manufacturing and allows for further exploration of network architecture and training strategy with few training samples for single-shot 3D measurement.https://www.mdpi.com/2304-6732/10/4/4173D measurementfringe projectionstructured lightdeep learningsingle-shot |
| spellingShingle | Mingzhu Wan Lingbao Kong Xing Peng Single-Shot Three-Dimensional Measurement by Fringe Analysis Network Photonics 3D measurement fringe projection structured light deep learning single-shot |
| title | Single-Shot Three-Dimensional Measurement by Fringe Analysis Network |
| title_full | Single-Shot Three-Dimensional Measurement by Fringe Analysis Network |
| title_fullStr | Single-Shot Three-Dimensional Measurement by Fringe Analysis Network |
| title_full_unstemmed | Single-Shot Three-Dimensional Measurement by Fringe Analysis Network |
| title_short | Single-Shot Three-Dimensional Measurement by Fringe Analysis Network |
| title_sort | single shot three dimensional measurement by fringe analysis network |
| topic | 3D measurement fringe projection structured light deep learning single-shot |
| url | https://www.mdpi.com/2304-6732/10/4/417 |
| work_keys_str_mv | AT mingzhuwan singleshotthreedimensionalmeasurementbyfringeanalysisnetwork AT lingbaokong singleshotthreedimensionalmeasurementbyfringeanalysisnetwork AT xingpeng singleshotthreedimensionalmeasurementbyfringeanalysisnetwork |