Differentiable Vector Graphics Rasterization for Editing and Learning
We introduce a differentiable rasterizer that bridges the vector graphics and raster image domains, enabling powerful raster-based loss functions, optimization procedures, and machine learning techniques to edit and generate vector content. We observe that vector graphics rasterization is differenti...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
ACM|SIGGRAPH Asia 2020 Technical Papers
2025
|
| Online Access: | https://hdl.handle.net/1721.1/158158 |
| _version_ | 1824457868332498944 |
|---|---|
| author | Li, Tzu-Mao Lukac, Mike Gharbi, Michael Ragan-Kelley, Jonathan |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Li, Tzu-Mao Lukac, Mike Gharbi, Michael Ragan-Kelley, Jonathan |
| author_sort | Li, Tzu-Mao |
| collection | MIT |
| description | We introduce a differentiable rasterizer that bridges the vector graphics and raster image domains, enabling powerful raster-based loss functions, optimization procedures, and machine learning techniques to edit and generate vector content. We observe that vector graphics rasterization is differentiable after pixel prefiltering. Our differentiable rasterizer offers two prefiltering options: an analytical prefiltering technique and a multisampling anti-aliasing technique. The analytical variant is faster but can suffer from artifacts such as conflation. The multisampling variant is still efficient, and can render high-quality images while computing unbiased gradients for each pixel with respect to curve parameters.
We demonstrate that our rasterizer enables new applications, including a vector graphics editor guided by image metrics, a painterly rendering algorithm that fits vector primitives to an image by minimizing a deep perceptual loss function, new vector graphics editing algorithms that exploit well-known image processing methods such as seam carving, and deep generative models that generate vector content from raster-only supervision under a VAE or GAN training objective. |
| first_indexed | 2025-02-19T04:16:50Z |
| format | Article |
| id | mit-1721.1/158158 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2025-02-19T04:16:50Z |
| publishDate | 2025 |
| publisher | ACM|SIGGRAPH Asia 2020 Technical Papers |
| record_format | dspace |
| spelling | mit-1721.1/1581582025-02-03T17:05:55Z Differentiable Vector Graphics Rasterization for Editing and Learning Li, Tzu-Mao Lukac, Mike Gharbi, Michael Ragan-Kelley, Jonathan Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory We introduce a differentiable rasterizer that bridges the vector graphics and raster image domains, enabling powerful raster-based loss functions, optimization procedures, and machine learning techniques to edit and generate vector content. We observe that vector graphics rasterization is differentiable after pixel prefiltering. Our differentiable rasterizer offers two prefiltering options: an analytical prefiltering technique and a multisampling anti-aliasing technique. The analytical variant is faster but can suffer from artifacts such as conflation. The multisampling variant is still efficient, and can render high-quality images while computing unbiased gradients for each pixel with respect to curve parameters. We demonstrate that our rasterizer enables new applications, including a vector graphics editor guided by image metrics, a painterly rendering algorithm that fits vector primitives to an image by minimizing a deep perceptual loss function, new vector graphics editing algorithms that exploit well-known image processing methods such as seam carving, and deep generative models that generate vector content from raster-only supervision under a VAE or GAN training objective. 2025-02-03T17:05:53Z 2025-02-03T17:05:53Z 2020-11-26 2025-02-01T08:51:32Z Article http://purl.org/eprint/type/JournalArticle 978-1-4503-8107-9 https://hdl.handle.net/1721.1/158158 Li, Tzu-Mao, Lukac, Mike, Gharbi, Michael and Ragan-Kelley, Jonathan. 2020. "Differentiable Vector Graphics Rasterization for Editing and Learning." ACM Transactions on Graphics, 39 (6). PUBLISHER_POLICY en https://doi.org/10.1145/3414685.3417871 ACM Transactions on Graphics Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. The author(s) application/pdf ACM|SIGGRAPH Asia 2020 Technical Papers Association for Computing Machinery |
| spellingShingle | Li, Tzu-Mao Lukac, Mike Gharbi, Michael Ragan-Kelley, Jonathan Differentiable Vector Graphics Rasterization for Editing and Learning |
| title | Differentiable Vector Graphics Rasterization for Editing and Learning |
| title_full | Differentiable Vector Graphics Rasterization for Editing and Learning |
| title_fullStr | Differentiable Vector Graphics Rasterization for Editing and Learning |
| title_full_unstemmed | Differentiable Vector Graphics Rasterization for Editing and Learning |
| title_short | Differentiable Vector Graphics Rasterization for Editing and Learning |
| title_sort | differentiable vector graphics rasterization for editing and learning |
| url | https://hdl.handle.net/1721.1/158158 |
| work_keys_str_mv | AT litzumao differentiablevectorgraphicsrasterizationforeditingandlearning AT lukacmike differentiablevectorgraphicsrasterizationforeditingandlearning AT gharbimichael differentiablevectorgraphicsrasterizationforeditingandlearning AT ragankelleyjonathan differentiablevectorgraphicsrasterizationforeditingandlearning |