Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions
Depth cameras are closely related to our daily lives and have been widely used in fields such as machine vision, autonomous driving, and virtual reality. Despite their diverse applications, depth cameras still encounter challenges like multi-path interference and mixed pixels. Compared to traditiona...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Photonics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2304-6732/10/11/1223 |
| _version_ | 1827639015632797696 |
|---|---|
| author | Tian-Long Wang Lin Ao Jie Zheng Zhi-Bin Sun |
| author_facet | Tian-Long Wang Lin Ao Jie Zheng Zhi-Bin Sun |
| author_sort | Tian-Long Wang |
| collection | DOAJ |
| description | Depth cameras are closely related to our daily lives and have been widely used in fields such as machine vision, autonomous driving, and virtual reality. Despite their diverse applications, depth cameras still encounter challenges like multi-path interference and mixed pixels. Compared to traditional sensors, depth cameras have lower resolution and a lower signal-to-noise ratio. Moreover, when used in environments with scattering media, object information scatters multiple times, making it difficult for time-of-flight (ToF) cameras to obtain effective object data. To tackle these issues, we propose a solution that combines ToF cameras with second-order correlation transform theory. In this article, we explore the utilization of ToF camera depth information within a computational correlated imaging system under ambient light conditions. We integrate compressed sensing and non-training neural networks with ToF technology to reconstruct depth images from a series of measurements at a low sampling rate. The research indicates that by leveraging the depth data collected by the camera, we can recover negative depth images. We analyzed and addressed the reasons behind the generation of negative depth images. Additionally, under undersampling conditions, the use of reconstruction algorithms results in a higher peak signal-to-noise ratio compared to images obtained from the original camera. The results demonstrate that the introduced second-order correlation transformation can effectively reduce noise originating from the ToF camera itself and direct ambient light, thereby enabling the use of ToF cameras in complex environments such as scattering media. |
| first_indexed | 2024-03-09T16:31:48Z |
| format | Article |
| id | doaj.art-da3579e11f8341f0a5880b8b7f4dfff4 |
| institution | Directory Open Access Journal |
| issn | 2304-6732 |
| language | English |
| last_indexed | 2024-03-09T16:31:48Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Photonics |
| spelling | doaj.art-da3579e11f8341f0a5880b8b7f4dfff42023-11-24T15:01:26ZengMDPI AGPhotonics2304-67322023-10-011011122310.3390/photonics10111223Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation FunctionsTian-Long Wang0Lin Ao1Jie Zheng2Zhi-Bin Sun3National Space Science Center, Chinese Academy of Sciences, Beijing 100190, ChinaNortheastern University, Shenyang 110819, ChinaNortheastern University, Shenyang 110819, ChinaNational Space Science Center, Chinese Academy of Sciences, Beijing 100190, ChinaDepth cameras are closely related to our daily lives and have been widely used in fields such as machine vision, autonomous driving, and virtual reality. Despite their diverse applications, depth cameras still encounter challenges like multi-path interference and mixed pixels. Compared to traditional sensors, depth cameras have lower resolution and a lower signal-to-noise ratio. Moreover, when used in environments with scattering media, object information scatters multiple times, making it difficult for time-of-flight (ToF) cameras to obtain effective object data. To tackle these issues, we propose a solution that combines ToF cameras with second-order correlation transform theory. In this article, we explore the utilization of ToF camera depth information within a computational correlated imaging system under ambient light conditions. We integrate compressed sensing and non-training neural networks with ToF technology to reconstruct depth images from a series of measurements at a low sampling rate. The research indicates that by leveraging the depth data collected by the camera, we can recover negative depth images. We analyzed and addressed the reasons behind the generation of negative depth images. Additionally, under undersampling conditions, the use of reconstruction algorithms results in a higher peak signal-to-noise ratio compared to images obtained from the original camera. The results demonstrate that the introduced second-order correlation transformation can effectively reduce noise originating from the ToF camera itself and direct ambient light, thereby enabling the use of ToF cameras in complex environments such as scattering media.https://www.mdpi.com/2304-6732/10/11/1223time-of-flightcomputational correlation imagingscattering mediacompressed sensinguntrained neural network |
| spellingShingle | Tian-Long Wang Lin Ao Jie Zheng Zhi-Bin Sun Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions Photonics time-of-flight computational correlation imaging scattering media compressed sensing untrained neural network |
| title | Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions |
| title_full | Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions |
| title_fullStr | Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions |
| title_full_unstemmed | Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions |
| title_short | Reconstructing Depth Images for Time-of-Flight Cameras Based on Second-Order Correlation Functions |
| title_sort | reconstructing depth images for time of flight cameras based on second order correlation functions |
| topic | time-of-flight computational correlation imaging scattering media compressed sensing untrained neural network |
| url | https://www.mdpi.com/2304-6732/10/11/1223 |
| work_keys_str_mv | AT tianlongwang reconstructingdepthimagesfortimeofflightcamerasbasedonsecondordercorrelationfunctions AT linao reconstructingdepthimagesfortimeofflightcamerasbasedonsecondordercorrelationfunctions AT jiezheng reconstructingdepthimagesfortimeofflightcamerasbasedonsecondordercorrelationfunctions AT zhibinsun reconstructingdepthimagesfortimeofflightcamerasbasedonsecondordercorrelationfunctions |