Hierarchical vectorization for facial images
The explosive growth of social media means portrait editing and retouching are in high demand. While portraits are commonly captured and stored as raster images, editing raster images is non-trivial and requires the user to be highly skilled. Aiming at developing intuitive and easy-to-use portrait e...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/173109 |
| _version_ | 1811682626356903936 |
|---|---|
| author | Fu, Qian Liu, Linlin Hou, Fei He, Ying |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Fu, Qian Liu, Linlin Hou, Fei He, Ying |
| author_sort | Fu, Qian |
| collection | NTU |
| description | The explosive growth of social media means portrait editing and retouching are in high demand. While portraits are commonly captured and stored as raster images, editing raster images is non-trivial and requires the user to be highly skilled. Aiming at developing intuitive and easy-to-use portrait editing tools, we propose a novel vectorization method that can automatically convert raster images into a 3-tier hierarchical representation. The base layer consists of a set of sparse diffusion curves (DCs) which characterize salient geometric features and low-frequency colors, providing a means for semantic color transfer and facial expression editing. The middle level encodes specular highlights and shadows as large, editable Poisson regions (PRs) and allows the user to directly adjust illumination by tuning the strength and changing the shapes of PRs. The top level contains two types of pixel-sized PRs for high-frequency residuals and fine details such as pimples and pigmentation. We train a deep generative model that can produce high-frequency residuals automatically. Thanks to the inherent meaning in vector primitives, editing portraits becomes easy and intuitive. In particular, our method supports color transfer, facial expression editing, highlight and shadow editing, and automatic retouching. To quantitatively evaluate the results, we extend the commonly used FLIP metric (which measures color and feature differences between two images) to consider illumination. The new metric, illumination-sensitive FLIP, can effectively capture salient changes in color transfer results, and is more consistent with human perception than FLIP and other quality measures for portrait images. We evaluate our method on the FFHQR dataset and show it to be effective for common portrait editing tasks, such as retouching, light editing, color transfer, and expression editing.[Figure not available: see fulltext.]. |
| first_indexed | 2024-10-01T03:59:50Z |
| format | Journal Article |
| id | ntu-10356/173109 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:59:50Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1731092024-01-12T15:36:59Z Hierarchical vectorization for facial images Fu, Qian Liu, Linlin Hou, Fei He, Ying School of Computer Science and Engineering Engineering::Civil engineering Face Editing Color Transfer The explosive growth of social media means portrait editing and retouching are in high demand. While portraits are commonly captured and stored as raster images, editing raster images is non-trivial and requires the user to be highly skilled. Aiming at developing intuitive and easy-to-use portrait editing tools, we propose a novel vectorization method that can automatically convert raster images into a 3-tier hierarchical representation. The base layer consists of a set of sparse diffusion curves (DCs) which characterize salient geometric features and low-frequency colors, providing a means for semantic color transfer and facial expression editing. The middle level encodes specular highlights and shadows as large, editable Poisson regions (PRs) and allows the user to directly adjust illumination by tuning the strength and changing the shapes of PRs. The top level contains two types of pixel-sized PRs for high-frequency residuals and fine details such as pimples and pigmentation. We train a deep generative model that can produce high-frequency residuals automatically. Thanks to the inherent meaning in vector primitives, editing portraits becomes easy and intuitive. In particular, our method supports color transfer, facial expression editing, highlight and shadow editing, and automatic retouching. To quantitatively evaluate the results, we extend the commonly used FLIP metric (which measures color and feature differences between two images) to consider illumination. The new metric, illumination-sensitive FLIP, can effectively capture salient changes in color transfer results, and is more consistent with human perception than FLIP and other quality measures for portrait images. We evaluate our method on the FFHQR dataset and show it to be effective for common portrait editing tasks, such as retouching, light editing, color transfer, and expression editing.[Figure not available: see fulltext.]. Ministry of Defence (MINDEF) Published version This project was supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 1 (RG20/20), the National Natural Science Foundation of China (61872347), and the Special Plan for the Development of Distinguished Young Scientists of ISCAS (Y8RC535018). 2024-01-12T06:37:58Z 2024-01-12T06:37:58Z 2023 Journal Article Fu, Q., Liu, L., Hou, F. & He, Y. (2023). Hierarchical vectorization for facial images. Computational Visual Media, 10(1), 97-118. https://dx.doi.org/10.1007/s41095-022-0314-4 2096-0433 https://hdl.handle.net/10356/173109 10.1007/s41095-022-0314-4 2-s2.0-85178226411 1 10 97 118 en RG20/20 Computational Visual Media ⃝© The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate cred it to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. application/pdf |
| spellingShingle | Engineering::Civil engineering Face Editing Color Transfer Fu, Qian Liu, Linlin Hou, Fei He, Ying Hierarchical vectorization for facial images |
| title | Hierarchical vectorization for facial images |
| title_full | Hierarchical vectorization for facial images |
| title_fullStr | Hierarchical vectorization for facial images |
| title_full_unstemmed | Hierarchical vectorization for facial images |
| title_short | Hierarchical vectorization for facial images |
| title_sort | hierarchical vectorization for facial images |
| topic | Engineering::Civil engineering Face Editing Color Transfer |
| url | https://hdl.handle.net/10356/173109 |
| work_keys_str_mv | AT fuqian hierarchicalvectorizationforfacialimages AT liulinlin hierarchicalvectorizationforfacialimages AT houfei hierarchicalvectorizationforfacialimages AT heying hierarchicalvectorizationforfacialimages |