Semantic photo manipulation with a generative image prior
© 2019 Association for Computing Machinery. All rights reserved. Despite the recent success of GANs in synthesizing images conditioned on inputs such as a user sketch, text, or semantic labels, manipulating the highlevel attributes of an existing natural photograph with GANs is challenging for two r...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Association for Computing Machinery (ACM)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/136406 |
| _version_ | 1811097093322833920 |
|---|---|
| author | Bau, David Strobelt, Hendrik Peebles, William Wulff, Jonas Zhou, Bolei Zhu, Jun-Yan Torralba, Antonio |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Bau, David Strobelt, Hendrik Peebles, William Wulff, Jonas Zhou, Bolei Zhu, Jun-Yan Torralba, Antonio |
| author_sort | Bau, David |
| collection | MIT |
| description | © 2019 Association for Computing Machinery. All rights reserved. Despite the recent success of GANs in synthesizing images conditioned on inputs such as a user sketch, text, or semantic labels, manipulating the highlevel attributes of an existing natural photograph with GANs is challenging for two reasons. First, it is hard for GANs to precisely reproduce an input image. Second, after manipulation, the newly synthesized pixels often do not fit the original image. In this paper, we address these issues by adapting the image prior learned by GANs to image statistics of an individual image. Our method can accurately reconstruct the input image and synthesize new content, consistent with the appearance of the input image. We demonstrate our interactive system on several semantic image editing tasks, including synthesizing new objects consistent with background, removing unwanted objects, and changing the appearance of an object. Quantitative and qualitative comparisons against several existing methods demonstrate the effectiveness of our method. |
| first_indexed | 2024-09-23T16:54:13Z |
| format | Article |
| id | mit-1721.1/136406 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T16:54:13Z |
| publishDate | 2021 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/1364062023-01-20T20:42:19Z Semantic photo manipulation with a generative image prior Bau, David Strobelt, Hendrik Peebles, William Wulff, Jonas Zhou, Bolei Zhu, Jun-Yan Torralba, Antonio Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory MIT-IBM Watson AI Lab © 2019 Association for Computing Machinery. All rights reserved. Despite the recent success of GANs in synthesizing images conditioned on inputs such as a user sketch, text, or semantic labels, manipulating the highlevel attributes of an existing natural photograph with GANs is challenging for two reasons. First, it is hard for GANs to precisely reproduce an input image. Second, after manipulation, the newly synthesized pixels often do not fit the original image. In this paper, we address these issues by adapting the image prior learned by GANs to image statistics of an individual image. Our method can accurately reconstruct the input image and synthesize new content, consistent with the appearance of the input image. We demonstrate our interactive system on several semantic image editing tasks, including synthesizing new objects consistent with background, removing unwanted objects, and changing the appearance of an object. Quantitative and qualitative comparisons against several existing methods demonstrate the effectiveness of our method. 2021-10-27T20:35:13Z 2021-10-27T20:35:13Z 2019 2021-04-15T17:32:39Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/136406 en 10.1145/3306346.3323023 ACM Transactions on Graphics Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Computing Machinery (ACM) arXiv |
| spellingShingle | Bau, David Strobelt, Hendrik Peebles, William Wulff, Jonas Zhou, Bolei Zhu, Jun-Yan Torralba, Antonio Semantic photo manipulation with a generative image prior |
| title | Semantic photo manipulation with a generative image prior |
| title_full | Semantic photo manipulation with a generative image prior |
| title_fullStr | Semantic photo manipulation with a generative image prior |
| title_full_unstemmed | Semantic photo manipulation with a generative image prior |
| title_short | Semantic photo manipulation with a generative image prior |
| title_sort | semantic photo manipulation with a generative image prior |
| url | https://hdl.handle.net/1721.1/136406 |
| work_keys_str_mv | AT baudavid semanticphotomanipulationwithagenerativeimageprior AT strobelthendrik semanticphotomanipulationwithagenerativeimageprior AT peebleswilliam semanticphotomanipulationwithagenerativeimageprior AT wulffjonas semanticphotomanipulationwithagenerativeimageprior AT zhoubolei semanticphotomanipulationwithagenerativeimageprior AT zhujunyan semanticphotomanipulationwithagenerativeimageprior AT torralbaantonio semanticphotomanipulationwithagenerativeimageprior |