Physics-informed deep learning for fringe pattern analysis
Recently, deep learning has yielded transformative success across optics and photonics, especially in optical metrology. Deep neural networks (DNNs) with a fully convolutional architecture (e.g., U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various op...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Institue of Optics and Electronics, Chinese Academy of Sciences
2024-01-01
|
| Series: | Opto-Electronic Advances |
| Subjects: | |
| Online Access: | https://www.oejournal.org/article/doi/10.29026/oea.2024.230034 |
| _version_ | 1797262904690475008 |
|---|---|
| author | Wei Yin Yuxuan Che Xinsheng Li Mingyu Li Yan Hu Shijie Feng Edmund Y. Lam Qian Chen Chao Zuo |
| author_facet | Wei Yin Yuxuan Che Xinsheng Li Mingyu Li Yan Hu Shijie Feng Edmund Y. Lam Qian Chen Chao Zuo |
| author_sort | Wei Yin |
| collection | DOAJ |
| description | Recently, deep learning has yielded transformative success across optics and photonics, especially in optical metrology. Deep neural networks (DNNs) with a fully convolutional architecture (e.g., U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various optical metrology tasks, such as fringe denoising, phase unwrapping, and fringe analysis. However, the task of training a DNN to accurately identify an image-to-image transform from massive input and output data pairs seems at best naïve, as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice. To this end, we introduce a physics-informed deep learning method for fringe pattern analysis (PI-FPA) to overcome this limit by integrating a lightweight DNN with a learning-enhanced Fourier transform profilometry (LeFTP) module. By parameterizing conventional phase retrieval methods, the LeFTP module embeds the prior knowledge in the network structure and the loss function to directly provide reliable phase results for new types of samples, while circumventing the requirement of collecting a large amount of high-quality data in supervised learning methods. Guided by the initial phase from LeFTP, the phase recovery ability of the lightweight DNN is enhanced to further improve the phase accuracy at a low computational cost compared with existing end-to-end networks. Experimental results demonstrate that PI-FPA enables more accurate and computationally efficient single-shot phase retrieval, exhibiting its excellent generalization to various unseen objects during training. The proposed PI-FPA presents that challenging issues in optical metrology can be potentially overcome through the synergy of physics-priors-based traditional tools and data-driven learning approaches, opening new avenues to achieve fast and accurate single-shot 3D imaging. |
| first_indexed | 2024-04-25T00:04:32Z |
| format | Article |
| id | doaj.art-1dd446060d2c4d5e937400dc8e6c693a |
| institution | Directory Open Access Journal |
| issn | 2096-4579 |
| language | English |
| last_indexed | 2024-04-25T00:04:32Z |
| publishDate | 2024-01-01 |
| publisher | Institue of Optics and Electronics, Chinese Academy of Sciences |
| record_format | Article |
| series | Opto-Electronic Advances |
| spelling | doaj.art-1dd446060d2c4d5e937400dc8e6c693a2024-03-14T06:14:23ZengInstitue of Optics and Electronics, Chinese Academy of SciencesOpto-Electronic Advances2096-45792024-01-017111210.29026/oea.2024.230034OEA-2023-0034ZuochaoPhysics-informed deep learning for fringe pattern analysisWei Yin0Yuxuan Che1Xinsheng Li2Mingyu Li3Yan Hu4Shijie Feng5Edmund Y. Lam6Qian Chen7Chao Zuo8Smart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSmart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSmart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSmart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSmart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaSmart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaDepartment of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam, Hong Kong SAR 999077, ChinaJiangsu Key Laboratory of Spectral Imaging Intelligent Sense, Nanjing 210094, ChinaSmart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaRecently, deep learning has yielded transformative success across optics and photonics, especially in optical metrology. Deep neural networks (DNNs) with a fully convolutional architecture (e.g., U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various optical metrology tasks, such as fringe denoising, phase unwrapping, and fringe analysis. However, the task of training a DNN to accurately identify an image-to-image transform from massive input and output data pairs seems at best naïve, as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice. To this end, we introduce a physics-informed deep learning method for fringe pattern analysis (PI-FPA) to overcome this limit by integrating a lightweight DNN with a learning-enhanced Fourier transform profilometry (LeFTP) module. By parameterizing conventional phase retrieval methods, the LeFTP module embeds the prior knowledge in the network structure and the loss function to directly provide reliable phase results for new types of samples, while circumventing the requirement of collecting a large amount of high-quality data in supervised learning methods. Guided by the initial phase from LeFTP, the phase recovery ability of the lightweight DNN is enhanced to further improve the phase accuracy at a low computational cost compared with existing end-to-end networks. Experimental results demonstrate that PI-FPA enables more accurate and computationally efficient single-shot phase retrieval, exhibiting its excellent generalization to various unseen objects during training. The proposed PI-FPA presents that challenging issues in optical metrology can be potentially overcome through the synergy of physics-priors-based traditional tools and data-driven learning approaches, opening new avenues to achieve fast and accurate single-shot 3D imaging.https://www.oejournal.org/article/doi/10.29026/oea.2024.230034optical metrologydeep learningphysics-informed neural networksfringe analysisphase retrieval |
| spellingShingle | Wei Yin Yuxuan Che Xinsheng Li Mingyu Li Yan Hu Shijie Feng Edmund Y. Lam Qian Chen Chao Zuo Physics-informed deep learning for fringe pattern analysis Opto-Electronic Advances optical metrology deep learning physics-informed neural networks fringe analysis phase retrieval |
| title | Physics-informed deep learning for fringe pattern analysis |
| title_full | Physics-informed deep learning for fringe pattern analysis |
| title_fullStr | Physics-informed deep learning for fringe pattern analysis |
| title_full_unstemmed | Physics-informed deep learning for fringe pattern analysis |
| title_short | Physics-informed deep learning for fringe pattern analysis |
| title_sort | physics informed deep learning for fringe pattern analysis |
| topic | optical metrology deep learning physics-informed neural networks fringe analysis phase retrieval |
| url | https://www.oejournal.org/article/doi/10.29026/oea.2024.230034 |
| work_keys_str_mv | AT weiyin physicsinformeddeeplearningforfringepatternanalysis AT yuxuanche physicsinformeddeeplearningforfringepatternanalysis AT xinshengli physicsinformeddeeplearningforfringepatternanalysis AT mingyuli physicsinformeddeeplearningforfringepatternanalysis AT yanhu physicsinformeddeeplearningforfringepatternanalysis AT shijiefeng physicsinformeddeeplearningforfringepatternanalysis AT edmundylam physicsinformeddeeplearningforfringepatternanalysis AT qianchen physicsinformeddeeplearningforfringepatternanalysis AT chaozuo physicsinformeddeeplearningforfringepatternanalysis |