Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography
Time-of-flight photography is a new imaging modality that combines the precise time-of-flight measurements of lidar with the wide field of view and high angular resolution of photography. The result of this combination is that time-of-flight cameras by nature capture videos of light as it flows thr...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/152657 https://orcid.org/0000-0002-9215-7245 |
| _version_ | 1826214811997831168 |
|---|---|
| author | Henley, Connor A. |
| author2 | Raskar, Ramesh |
| author_facet | Raskar, Ramesh Henley, Connor A. |
| author_sort | Henley, Connor A. |
| collection | MIT |
| description | Time-of-flight photography is a new imaging modality that combines the precise time-of-flight measurements of lidar with the wide field of view and high angular resolution of photography. The result of this combination is that time-of-flight cameras by nature capture videos of light as it flows through the photographed scene. Unlike regular photographs, time-of-flight photographs expose the sequence of how light interacts with a scene. Furthermore, the fixed and finite speed of light in free space tightly couples the dimensions of space and time, such that it is typically straightforward to determine precisely where and when observed scattering events occur.
In this thesis we present five new imaging methods that leverage the unique advantages of time-of-flight photography to analyze light transport within a scene and, through this analysis, infer scene properties. First, we show that shadows observed in light that has scattered exactly twice before returning to the camera can be used to infer the geometry of surfaces that are hidden from view. Second, we show how the time-of-flight of these two-bounce returns can be used to retrieve the visible scene’s geometry. Third, we propose an algorithm to retrieve the spatially varying BRDF of visible surfaces from two-bounce returns. Fourth, we show how multi-bounce returns measured with a time-of-flight camera can be used to unambiguously detect and localize specular surfaces like mirrors and windows. Finally, we show how time-resolved measurements of light that scatters many times within snow before returning to the camera can be used to infer the snow’s density and grain size, and the concentrations of impurities within the snow.
Our hope is that introducing several useful and practical applications of time-of-flight photography will spur interest and motivate further research into new applications and the fundamental principles of this exciting new technology. |
| first_indexed | 2024-09-23T16:11:49Z |
| format | Thesis |
| id | mit-1721.1/152657 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:11:49Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1526572023-11-03T03:05:00Z Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography Henley, Connor A. Raskar, Ramesh Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Time-of-flight photography is a new imaging modality that combines the precise time-of-flight measurements of lidar with the wide field of view and high angular resolution of photography. The result of this combination is that time-of-flight cameras by nature capture videos of light as it flows through the photographed scene. Unlike regular photographs, time-of-flight photographs expose the sequence of how light interacts with a scene. Furthermore, the fixed and finite speed of light in free space tightly couples the dimensions of space and time, such that it is typically straightforward to determine precisely where and when observed scattering events occur. In this thesis we present five new imaging methods that leverage the unique advantages of time-of-flight photography to analyze light transport within a scene and, through this analysis, infer scene properties. First, we show that shadows observed in light that has scattered exactly twice before returning to the camera can be used to infer the geometry of surfaces that are hidden from view. Second, we show how the time-of-flight of these two-bounce returns can be used to retrieve the visible scene’s geometry. Third, we propose an algorithm to retrieve the spatially varying BRDF of visible surfaces from two-bounce returns. Fourth, we show how multi-bounce returns measured with a time-of-flight camera can be used to unambiguously detect and localize specular surfaces like mirrors and windows. Finally, we show how time-resolved measurements of light that scatters many times within snow before returning to the camera can be used to infer the snow’s density and grain size, and the concentrations of impurities within the snow. Our hope is that introducing several useful and practical applications of time-of-flight photography will spur interest and motivate further research into new applications and the fundamental principles of this exciting new technology. Ph.D. 2023-11-02T20:06:20Z 2023-11-02T20:06:20Z 2023-09 2023-09-21T14:25:52.750Z Thesis https://hdl.handle.net/1721.1/152657 https://orcid.org/0000-0002-9215-7245 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Henley, Connor A. Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography |
| title | Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography |
| title_full | Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography |
| title_fullStr | Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography |
| title_full_unstemmed | Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography |
| title_short | Shadows, Mirrors, and Snow: Decoding Multibounce Light Transport in Time-of-Flight Photography |
| title_sort | shadows mirrors and snow decoding multibounce light transport in time of flight photography |
| url | https://hdl.handle.net/1721.1/152657 https://orcid.org/0000-0002-9215-7245 |
| work_keys_str_mv | AT henleyconnora shadowsmirrorsandsnowdecodingmultibouncelighttransportintimeofflightphotography |