Data-driven hallucination of different times of day from a single outdoor photo
We introduce "time hallucination": synthesizing a plausible image at a different time of day from an input image. This challenging task often requires dramatically altering the color appearance of the picture. In this paper, we introduce the first data-driven approach to automatically crea...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery
2014
|
| Online Access: | http://hdl.handle.net/1721.1/86234 https://orcid.org/0000-0001-9919-069X https://orcid.org/0000-0002-2231-7995 |
| _version_ | 1826213418826203136 |
|---|---|
| author | Shih, YiChang Paris, Sylvain Durand, Fredo Freeman, William T. |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Shih, YiChang Paris, Sylvain Durand, Fredo Freeman, William T. |
| author_sort | Shih, YiChang |
| collection | MIT |
| description | We introduce "time hallucination": synthesizing a plausible image at a different time of day from an input image. This challenging task often requires dramatically altering the color appearance of the picture. In this paper, we introduce the first data-driven approach to automatically creating a plausible-looking photo that appears as though it were taken at a different time of day. The time of day is specified by a semantic time label, such as "night".
Our approach relies on a database of time-lapse videos of various scenes. These videos provide rich information about the variations in color appearance of a scene throughout the day. Our method transfers the color appearance from videos with a similar scene as the input photo. We propose a locally affine model learned from the video for the transfer, allowing our model to synthesize new color data while retaining image details. We show that this model can hallucinate a wide range of different times of day. The model generates a large sparse linear system, which can be solved by off-the-shelf solvers. We validate our methods by synthesizing transforming photos of various outdoor scenes to four times of interest: daytime, the golden hour, the blue hour, and nighttime. |
| first_indexed | 2024-09-23T15:48:47Z |
| format | Article |
| id | mit-1721.1/86234 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:48:47Z |
| publishDate | 2014 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | mit-1721.1/862342022-10-02T04:16:09Z Data-driven hallucination of different times of day from a single outdoor photo Shih, YiChang Paris, Sylvain Durand, Fredo Freeman, William T. Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Shih, YiChang Durand, Fredo Freeman, William T. We introduce "time hallucination": synthesizing a plausible image at a different time of day from an input image. This challenging task often requires dramatically altering the color appearance of the picture. In this paper, we introduce the first data-driven approach to automatically creating a plausible-looking photo that appears as though it were taken at a different time of day. The time of day is specified by a semantic time label, such as "night". Our approach relies on a database of time-lapse videos of various scenes. These videos provide rich information about the variations in color appearance of a scene throughout the day. Our method transfers the color appearance from videos with a similar scene as the input photo. We propose a locally affine model learned from the video for the transfer, allowing our model to synthesize new color data while retaining image details. We show that this model can hallucinate a wide range of different times of day. The model generates a large sparse linear system, which can be solved by off-the-shelf solvers. We validate our methods by synthesizing transforming photos of various outdoor scenes to four times of interest: daytime, the golden hour, the blue hour, and nighttime. National Science Foundation (U.S.) (NSF No.0964004) National Science Foundation (U.S.) (NSF CGV-1111415) 2014-04-24T19:02:54Z 2014-04-24T19:02:54Z 2013-11 Article http://purl.org/eprint/type/ConferencePaper 07300301 http://hdl.handle.net/1721.1/86234 Shih, Yichang, Sylvain Paris, Frédo Durand, and William T. Freeman. “Data-Driven Hallucination of Different Times of Day from a Single Outdoor Photo.” ACM Transactions on Graphics 32, no. 6 (November 1, 2013): 1–11. https://orcid.org/0000-0001-9919-069X https://orcid.org/0000-0002-2231-7995 en_US http://dx.doi.org/10.1145/2508363.2508419 ACM Transactions on Graphics Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Computing Machinery MIT web domain |
| spellingShingle | Shih, YiChang Paris, Sylvain Durand, Fredo Freeman, William T. Data-driven hallucination of different times of day from a single outdoor photo |
| title | Data-driven hallucination of different times of day from a single outdoor photo |
| title_full | Data-driven hallucination of different times of day from a single outdoor photo |
| title_fullStr | Data-driven hallucination of different times of day from a single outdoor photo |
| title_full_unstemmed | Data-driven hallucination of different times of day from a single outdoor photo |
| title_short | Data-driven hallucination of different times of day from a single outdoor photo |
| title_sort | data driven hallucination of different times of day from a single outdoor photo |
| url | http://hdl.handle.net/1721.1/86234 https://orcid.org/0000-0001-9919-069X https://orcid.org/0000-0002-2231-7995 |
| work_keys_str_mv | AT shihyichang datadrivenhallucinationofdifferenttimesofdayfromasingleoutdoorphoto AT parissylvain datadrivenhallucinationofdifferenttimesofdayfromasingleoutdoorphoto AT durandfredo datadrivenhallucinationofdifferenttimesofdayfromasingleoutdoorphoto AT freemanwilliamt datadrivenhallucinationofdifferenttimesofdayfromasingleoutdoorphoto |