Exploiting Repetitions for Image-Based Rendering of Facades
© 2018 The Author(s) Computer Graphics Forum © 2018 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd. Street-level imagery is now abundant but does not have sufficient capture density to be usable for Image-Based Rendering (IBR) of facades. We pr...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135816 |
| _version_ | 1826213386500702208 |
|---|---|
| author | Rodriguez, Simon Bousseau, Adrien Durand, Fredo Drettakis, George |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Rodriguez, Simon Bousseau, Adrien Durand, Fredo Drettakis, George |
| author_sort | Rodriguez, Simon |
| collection | MIT |
| description | © 2018 The Author(s) Computer Graphics Forum © 2018 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd. Street-level imagery is now abundant but does not have sufficient capture density to be usable for Image-Based Rendering (IBR) of facades. We present a method that exploits repetitive elements in facades - such as windows - to perform data augmentation, in turn improving camera calibration, reconstructed geometry and overall rendering quality for IBR. The main intuition behind our approach is that a few views of several instances of an element provide similar information to many views of a single instance of that element. We first select similar instances of an element from 3–4 views of a facade and transform them into a common coordinate system, creating a “platonic” element. We use this common space to refine the camera calibration of each view of each instance and to reconstruct a 3D mesh of the element with multi-view stereo, that we regularize to obtain a piecewise-planar mesh aligned with dominant image contours. Observing the same element under multiple views also allows us to identify reflective areas - such as glass panels - which we use at rendering time to generate plausible reflections using an environment map. Our detailed 3D mesh, augmented set of views, and reflection mask enable image-based rendering of much higher quality than results obtained using the input images directly. |
| first_indexed | 2024-09-23T15:48:16Z |
| format | Article |
| id | mit-1721.1/135816 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T15:48:16Z |
| publishDate | 2021 |
| publisher | Wiley |
| record_format | dspace |
| spelling | mit-1721.1/1358162023-01-10T19:56:02Z Exploiting Repetitions for Image-Based Rendering of Facades Rodriguez, Simon Bousseau, Adrien Durand, Fredo Drettakis, George Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory © 2018 The Author(s) Computer Graphics Forum © 2018 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd. Street-level imagery is now abundant but does not have sufficient capture density to be usable for Image-Based Rendering (IBR) of facades. We present a method that exploits repetitive elements in facades - such as windows - to perform data augmentation, in turn improving camera calibration, reconstructed geometry and overall rendering quality for IBR. The main intuition behind our approach is that a few views of several instances of an element provide similar information to many views of a single instance of that element. We first select similar instances of an element from 3–4 views of a facade and transform them into a common coordinate system, creating a “platonic” element. We use this common space to refine the camera calibration of each view of each instance and to reconstruct a 3D mesh of the element with multi-view stereo, that we regularize to obtain a piecewise-planar mesh aligned with dominant image contours. Observing the same element under multiple views also allows us to identify reflective areas - such as glass panels - which we use at rendering time to generate plausible reflections using an environment map. Our detailed 3D mesh, augmented set of views, and reflection mask enable image-based rendering of much higher quality than results obtained using the input images directly. 2021-10-27T20:29:26Z 2021-10-27T20:29:26Z 2018 2019-05-29T13:25:32Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135816 en 10.1111/CGF.13480 Computer Graphics Forum Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Wiley Other repository |
| spellingShingle | Rodriguez, Simon Bousseau, Adrien Durand, Fredo Drettakis, George Exploiting Repetitions for Image-Based Rendering of Facades |
| title | Exploiting Repetitions for Image-Based Rendering of Facades |
| title_full | Exploiting Repetitions for Image-Based Rendering of Facades |
| title_fullStr | Exploiting Repetitions for Image-Based Rendering of Facades |
| title_full_unstemmed | Exploiting Repetitions for Image-Based Rendering of Facades |
| title_short | Exploiting Repetitions for Image-Based Rendering of Facades |
| title_sort | exploiting repetitions for image based rendering of facades |
| url | https://hdl.handle.net/1721.1/135816 |
| work_keys_str_mv | AT rodriguezsimon exploitingrepetitionsforimagebasedrenderingoffacades AT bousseauadrien exploitingrepetitionsforimagebasedrenderingoffacades AT durandfredo exploitingrepetitionsforimagebasedrenderingoffacades AT drettakisgeorge exploitingrepetitionsforimagebasedrenderingoffacades |