A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance
Abstract Whether in the realms of computer vision, robotics, or environmental monitoring, the ability to monitor and follow specific targets amidst intricate surroundings is essential for numerous applications. However, achieving rapid and efficient target tracking remains a challenge. Here we propo...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-03-01
|
| Series: | Communications Engineering |
| Online Access: | https://doi.org/10.1038/s44172-024-00191-7 |
| _version_ | 1797259209745629184 |
|---|---|
| author | Jiashuo Shi Taige Liu Liang Zhou Pei Yan Zhe Wang Xinyu Zhang |
| author_facet | Jiashuo Shi Taige Liu Liang Zhou Pei Yan Zhe Wang Xinyu Zhang |
| author_sort | Jiashuo Shi |
| collection | DOAJ |
| description | Abstract Whether in the realms of computer vision, robotics, or environmental monitoring, the ability to monitor and follow specific targets amidst intricate surroundings is essential for numerous applications. However, achieving rapid and efficient target tracking remains a challenge. Here we propose an optical implementation for rapid tracking with negligible digital post-processing, leveraging an all-optical information processing. This work combines a diffractive-based optical nerual network with a layered liquid crystal electrical addressing architecture, synergizing the parallel processing capabilities inherent in light propagation with liquid crystal dynamic adaptation mechanism. Through a one-time effort training, the trained network enable accurate prediction of the desired arrangement of liquid crystal molecules as confirmed through numerical blind testing. Then we establish an experimental camera architecture that synergistically combines an electrically-tuned functioned liquid crystal layer with materialized optical neural network. With integrating the architecture into optical imaging path of a detector plane, this optical computing camera offers a data-driven diffractive guidance, enabling the identification of target within complex backgrounds, highlighting its high-level vision task implementation and problem-solving capabilities. |
| first_indexed | 2024-04-24T23:05:48Z |
| format | Article |
| id | doaj.art-54dc716b0b7a40aeab1eddb8170b4315 |
| institution | Directory Open Access Journal |
| issn | 2731-3395 |
| language | English |
| last_indexed | 2024-04-24T23:05:48Z |
| publishDate | 2024-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Engineering |
| spelling | doaj.art-54dc716b0b7a40aeab1eddb8170b43152024-03-17T12:27:47ZengNature PortfolioCommunications Engineering2731-33952024-03-013111010.1038/s44172-024-00191-7A physics-informed deep learning liquid crystal camera with data-driven diffractive guidanceJiashuo Shi0Taige Liu1Liang Zhou2Pei Yan3Zhe Wang4Xinyu Zhang5National Key Laboratory of Science and Technology on Multi-spectral Information Processing, Huazhong University of Science and TechnologyNational Key Laboratory of Science and Technology on Multi-spectral Information Processing, Huazhong University of Science and TechnologyNational Key Laboratory of Science and Technology on Multi-spectral Information Processing, Huazhong University of Science and TechnologyNational Key Laboratory of Science and Technology on Multi-spectral Information Processing, Huazhong University of Science and TechnologyNational Key Laboratory of Science and Technology on Multi-spectral Information Processing, Huazhong University of Science and TechnologyNational Key Laboratory of Science and Technology on Multi-spectral Information Processing, Huazhong University of Science and TechnologyAbstract Whether in the realms of computer vision, robotics, or environmental monitoring, the ability to monitor and follow specific targets amidst intricate surroundings is essential for numerous applications. However, achieving rapid and efficient target tracking remains a challenge. Here we propose an optical implementation for rapid tracking with negligible digital post-processing, leveraging an all-optical information processing. This work combines a diffractive-based optical nerual network with a layered liquid crystal electrical addressing architecture, synergizing the parallel processing capabilities inherent in light propagation with liquid crystal dynamic adaptation mechanism. Through a one-time effort training, the trained network enable accurate prediction of the desired arrangement of liquid crystal molecules as confirmed through numerical blind testing. Then we establish an experimental camera architecture that synergistically combines an electrically-tuned functioned liquid crystal layer with materialized optical neural network. With integrating the architecture into optical imaging path of a detector plane, this optical computing camera offers a data-driven diffractive guidance, enabling the identification of target within complex backgrounds, highlighting its high-level vision task implementation and problem-solving capabilities.https://doi.org/10.1038/s44172-024-00191-7 |
| spellingShingle | Jiashuo Shi Taige Liu Liang Zhou Pei Yan Zhe Wang Xinyu Zhang A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance Communications Engineering |
| title | A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance |
| title_full | A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance |
| title_fullStr | A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance |
| title_full_unstemmed | A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance |
| title_short | A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance |
| title_sort | physics informed deep learning liquid crystal camera with data driven diffractive guidance |
| url | https://doi.org/10.1038/s44172-024-00191-7 |
| work_keys_str_mv | AT jiashuoshi aphysicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT taigeliu aphysicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT liangzhou aphysicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT peiyan aphysicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT zhewang aphysicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT xinyuzhang aphysicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT jiashuoshi physicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT taigeliu physicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT liangzhou physicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT peiyan physicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT zhewang physicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance AT xinyuzhang physicsinformeddeeplearningliquidcrystalcamerawithdatadrivendiffractiveguidance |