Video face replacement
We present a method for replacing facial performances in video. Our approach accounts for differences in identity, visual appearance, speech, and timing between source and target videos. Unlike prior work, it does not require substantial manual operation or complex acquisition hardware, only single-...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2012
|
| Online Access: | http://hdl.handle.net/1721.1/73421 https://orcid.org/0000-0003-0212-5643 |
| _version_ | 1826205612553273344 |
|---|---|
| author | Dale, Kevin Sunkavalli, Kalyan Johnson, Micah K. Vlasic, Daniel Matusik, Wojciech Pfister, Hanspeter |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Dale, Kevin Sunkavalli, Kalyan Johnson, Micah K. Vlasic, Daniel Matusik, Wojciech Pfister, Hanspeter |
| author_sort | Dale, Kevin |
| collection | MIT |
| description | We present a method for replacing facial performances in video. Our approach accounts for differences in identity, visual appearance, speech, and timing between source and target videos. Unlike prior work, it does not require substantial manual operation or complex acquisition hardware, only single-camera video. We use a 3D multilinear model to track the facial performance in both videos. Using the corresponding 3D geometry, we warp the source to the target face and retime the source to match the target performance. We then compute an optimal seam through the video volume that maintains temporal consistency in the final composite. We showcase the use of our method on a variety of examples and present the result of a user study that suggests our results are difficult to distinguish from real video footage. |
| first_indexed | 2024-09-23T13:15:51Z |
| format | Article |
| id | mit-1721.1/73421 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T13:15:51Z |
| publishDate | 2012 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/734212022-09-28T12:59:59Z Video face replacement Dale, Kevin Sunkavalli, Kalyan Johnson, Micah K. Vlasic, Daniel Matusik, Wojciech Pfister, Hanspeter Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Johnson, Micah K. Matusik, Wojciech We present a method for replacing facial performances in video. Our approach accounts for differences in identity, visual appearance, speech, and timing between source and target videos. Unlike prior work, it does not require substantial manual operation or complex acquisition hardware, only single-camera video. We use a 3D multilinear model to track the facial performance in both videos. Using the corresponding 3D geometry, we warp the source to the target face and retime the source to match the target performance. We then compute an optimal seam through the video volume that maintains temporal consistency in the final composite. We showcase the use of our method on a variety of examples and present the result of a user study that suggests our results are difficult to distinguish from real video footage. National Science Foundation (U.S.) (Grant PHY-0835713) National Science Foundation (U.S.) (Grant DMS-0739255) 2012-09-27T15:34:11Z 2012-09-27T15:34:11Z 2011-12 Article http://purl.org/eprint/type/ConferencePaper 978-1-4503-0807-6 http://hdl.handle.net/1721.1/73421 Kevin Dale, Kalyan Sunkavalli, Micah K. Johnson, Daniel Vlasic, Wojciech Matusik, and Hanspeter Pfister. 2011. Video face replacement. In Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11). ACM, New York, NY, USA, , Article 130 , 10 pages. https://orcid.org/0000-0003-0212-5643 en_US http://doi.acm.org/10.1145/2024156.2024164 Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11) Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Association for Computing Machinery (ACM) MIT web domain |
| spellingShingle | Dale, Kevin Sunkavalli, Kalyan Johnson, Micah K. Vlasic, Daniel Matusik, Wojciech Pfister, Hanspeter Video face replacement |
| title | Video face replacement |
| title_full | Video face replacement |
| title_fullStr | Video face replacement |
| title_full_unstemmed | Video face replacement |
| title_short | Video face replacement |
| title_sort | video face replacement |
| url | http://hdl.handle.net/1721.1/73421 https://orcid.org/0000-0003-0212-5643 |
| work_keys_str_mv | AT dalekevin videofacereplacement AT sunkavallikalyan videofacereplacement AT johnsonmicahk videofacereplacement AT vlasicdaniel videofacereplacement AT matusikwojciech videofacereplacement AT pfisterhanspeter videofacereplacement |