Novel View Synthesis from Casually Recorded Videos
Generating new, photorealistic views of a scene given only a single video is a difficult task that computer vision researchers have worked on for decades. This problem has recently seen a resurgence in interest due to its potential application in areas such as virtual reality. However, current novel...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/140103 |
| _version_ | 1826188908483837952 |
|---|---|
| author | Qian, Eric Ding |
| author2 | Freeman, William T. |
| author_facet | Freeman, William T. Qian, Eric Ding |
| author_sort | Qian, Eric Ding |
| collection | MIT |
| description | Generating new, photorealistic views of a scene given only a single video is a difficult task that computer vision researchers have worked on for decades. This problem has recently seen a resurgence in interest due to its potential application in areas such as virtual reality. However, current novel view synthesis techniques are not suitable for the short, casual videos that people typically record. Such videos deviate from the setups that these approaches typically use, where there are dense, high-resolution images of the scene. In this paper, we propose a method for refining an initial, coarse scene geometry which we then use for novel view synthesis on short video sequences. The core of our method is a geometry refinement step where we project the geometry to source views to remove inconsistent points. This refined geometry provides important shape and appearance information in data poor regions that would otherwise be difficult to accurately render. We evaluate our approach on the RealEstate10K dataset and demonstrate that compared to prior work, we synthesize views that are more temporally consistent. |
| first_indexed | 2024-09-23T08:06:34Z |
| format | Thesis |
| id | mit-1721.1/140103 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T08:06:34Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1401032022-02-08T04:07:08Z Novel View Synthesis from Casually Recorded Videos Qian, Eric Ding Freeman, William T. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Generating new, photorealistic views of a scene given only a single video is a difficult task that computer vision researchers have worked on for decades. This problem has recently seen a resurgence in interest due to its potential application in areas such as virtual reality. However, current novel view synthesis techniques are not suitable for the short, casual videos that people typically record. Such videos deviate from the setups that these approaches typically use, where there are dense, high-resolution images of the scene. In this paper, we propose a method for refining an initial, coarse scene geometry which we then use for novel view synthesis on short video sequences. The core of our method is a geometry refinement step where we project the geometry to source views to remove inconsistent points. This refined geometry provides important shape and appearance information in data poor regions that would otherwise be difficult to accurately render. We evaluate our approach on the RealEstate10K dataset and demonstrate that compared to prior work, we synthesize views that are more temporally consistent. M.Eng. 2022-02-07T15:24:15Z 2022-02-07T15:24:15Z 2021-09 2021-11-03T19:25:28.352Z Thesis https://hdl.handle.net/1721.1/140103 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Qian, Eric Ding Novel View Synthesis from Casually Recorded Videos |
| title | Novel View Synthesis from Casually Recorded Videos |
| title_full | Novel View Synthesis from Casually Recorded Videos |
| title_fullStr | Novel View Synthesis from Casually Recorded Videos |
| title_full_unstemmed | Novel View Synthesis from Casually Recorded Videos |
| title_short | Novel View Synthesis from Casually Recorded Videos |
| title_sort | novel view synthesis from casually recorded videos |
| url | https://hdl.handle.net/1721.1/140103 |
| work_keys_str_mv | AT qianericding novelviewsynthesisfromcasuallyrecordedvideos |