Modeling multi-style portrait relief from a single photograph
This paper aims at extending the method of Zhang et al. (2023) to produce not only portrait bas-reliefs from single photographs, but also high-depth reliefs with reasonable depth ordering. We cast this task as a problem of style-aware photo-to-depth translation, where the input is a photograph condi...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-12-01
|
| Series: | Graphical Models |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1524070323000401 |
| _version_ | 1797405514079928320 |
|---|---|
| author | Yu-Wei Zhang Hongguang Yang Ping Luo Zhi Li Hui Liu Zhongping Ji Caiming Zhang |
| author_facet | Yu-Wei Zhang Hongguang Yang Ping Luo Zhi Li Hui Liu Zhongping Ji Caiming Zhang |
| author_sort | Yu-Wei Zhang |
| collection | DOAJ |
| description | This paper aims at extending the method of Zhang et al. (2023) to produce not only portrait bas-reliefs from single photographs, but also high-depth reliefs with reasonable depth ordering. We cast this task as a problem of style-aware photo-to-depth translation, where the input is a photograph conditioned by a style vector and the output is a portrait relief with desired depth style. To construct ground-truth data for network training, we first propose an optimization-based method to synthesize high-depth reliefs from 3D portraits. Then, we train a normal-to-depth network to learn the mapping from normal maps to relief depths. After that, we use the trained network to generate high-depth relief samples using the provided normal maps from Zhang et al. (2023). As each normal map has pixel-wise photograph, we are able to establish correspondences between photographs and high-depth reliefs. By taking the bas-reliefs of Zhang et al. (2023), the new high-depth reliefs and their mixtures as target ground-truths, we finally train a encoder-to-decoder network to achieve style-aware relief modeling. Specially, the network is based on a U-shaped architecture, consisting of Swin Transformer blocks to process hierarchical deep features. Extensive experiments have demonstrated the effectiveness of the proposed method. Comparisons with previous works have verified its flexibility and state-of-the-art performance. |
| first_indexed | 2024-03-09T03:11:06Z |
| format | Article |
| id | doaj.art-8ee129289d304affbca76c319d41d871 |
| institution | Directory Open Access Journal |
| issn | 1524-0703 |
| language | English |
| last_indexed | 2024-03-09T03:11:06Z |
| publishDate | 2023-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Graphical Models |
| spelling | doaj.art-8ee129289d304affbca76c319d41d8712023-12-04T05:21:33ZengElsevierGraphical Models1524-07032023-12-01130101210Modeling multi-style portrait relief from a single photographYu-Wei Zhang0Hongguang Yang1Ping Luo2Zhi Li3Hui Liu4Zhongping Ji5Caiming Zhang6Faculty of Mechanical Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Corresponding author.Faculty of Mechanical Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, ChinaFaculty of Mechanical Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, ChinaFaculty of Mechanical Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, ChinaSchool of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, ChinaSchool of Computer Science, Hangzhou Dianzi University, Hangzhou, ChinaSchool of Computer Science and Technology, Shandong University, Jinan, ChinaThis paper aims at extending the method of Zhang et al. (2023) to produce not only portrait bas-reliefs from single photographs, but also high-depth reliefs with reasonable depth ordering. We cast this task as a problem of style-aware photo-to-depth translation, where the input is a photograph conditioned by a style vector and the output is a portrait relief with desired depth style. To construct ground-truth data for network training, we first propose an optimization-based method to synthesize high-depth reliefs from 3D portraits. Then, we train a normal-to-depth network to learn the mapping from normal maps to relief depths. After that, we use the trained network to generate high-depth relief samples using the provided normal maps from Zhang et al. (2023). As each normal map has pixel-wise photograph, we are able to establish correspondences between photographs and high-depth reliefs. By taking the bas-reliefs of Zhang et al. (2023), the new high-depth reliefs and their mixtures as target ground-truths, we finally train a encoder-to-decoder network to achieve style-aware relief modeling. Specially, the network is based on a U-shaped architecture, consisting of Swin Transformer blocks to process hierarchical deep features. Extensive experiments have demonstrated the effectiveness of the proposed method. Comparisons with previous works have verified its flexibility and state-of-the-art performance.http://www.sciencedirect.com/science/article/pii/S1524070323000401Portrait reliefMulti-style modelingSwin TransformerPhoto-to-depth translation |
| spellingShingle | Yu-Wei Zhang Hongguang Yang Ping Luo Zhi Li Hui Liu Zhongping Ji Caiming Zhang Modeling multi-style portrait relief from a single photograph Graphical Models Portrait relief Multi-style modeling Swin Transformer Photo-to-depth translation |
| title | Modeling multi-style portrait relief from a single photograph |
| title_full | Modeling multi-style portrait relief from a single photograph |
| title_fullStr | Modeling multi-style portrait relief from a single photograph |
| title_full_unstemmed | Modeling multi-style portrait relief from a single photograph |
| title_short | Modeling multi-style portrait relief from a single photograph |
| title_sort | modeling multi style portrait relief from a single photograph |
| topic | Portrait relief Multi-style modeling Swin Transformer Photo-to-depth translation |
| url | http://www.sciencedirect.com/science/article/pii/S1524070323000401 |
| work_keys_str_mv | AT yuweizhang modelingmultistyleportraitrelieffromasinglephotograph AT hongguangyang modelingmultistyleportraitrelieffromasinglephotograph AT pingluo modelingmultistyleportraitrelieffromasinglephotograph AT zhili modelingmultistyleportraitrelieffromasinglephotograph AT huiliu modelingmultistyleportraitrelieffromasinglephotograph AT zhongpingji modelingmultistyleportraitrelieffromasinglephotograph AT caimingzhang modelingmultistyleportraitrelieffromasinglephotograph |