MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis
Face sketch synthesis has made significant progress in the past few years. Recently, GAN-based methods have shown promising results on image-to-image translation problems, especially photo-to-sketch synthesis. Because the facial sketch has a hyper-abstract style and continuous graphic elements, comp...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9272961/ |
| _version_ | 1831678774552297472 |
|---|---|
| author | Kangning Du Huaqiang Zhou Lin Cao Yanan Guo Tao Wang |
| author_facet | Kangning Du Huaqiang Zhou Lin Cao Yanan Guo Tao Wang |
| author_sort | Kangning Du |
| collection | DOAJ |
| description | Face sketch synthesis has made significant progress in the past few years. Recently, GAN-based methods have shown promising results on image-to-image translation problems, especially photo-to-sketch synthesis. Because the facial sketch has a hyper-abstract style and continuous graphic elements, compared with other image styles, its local details are easier to expose small artifacts and blur. The existing face sketch synthesis methods lack models for specific facial regions and usually generate face sketches with coarse structures. To synthesis high-quality sketches and overcome the blurs and deformations, this paper proposes a novel Multi-Hierarchies GAN, which divides the face image into multiple hierarchical structures to learn different regions' features of the face. It includes three modules: a local region module, mask module, and fusion module. The local region module can learn the detailed features of different local regions of the face by GAN. The mask module can generate a coarse facial structure of a sketch and uses the facial feature extractor to enhance the high-level image and learn the latent spaces' feature. The fusion module can generate the final sketch by combining fine local regions and coarse facial structure. Extensive qualitative and quantitative experiments illustrate that the proposed method outperforms the state-of-the-art methods on the CUFS and CUFSF standard datasets and photos on the internet. |
| first_indexed | 2024-12-20T05:06:12Z |
| format | Article |
| id | doaj.art-a831157d1d124b1482a0476668648709 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T05:06:12Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a831157d1d124b1482a04766686487092022-12-21T19:52:23ZengIEEEIEEE Access2169-35362020-01-01821299521301110.1109/ACCESS.2020.30412849272961MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch SynthesisKangning Du0https://orcid.org/0000-0002-2998-757XHuaqiang Zhou1Lin Cao2https://orcid.org/0000-0003-0875-1549Yanan Guo3https://orcid.org/0000-0001-6366-3168Tao Wang4https://orcid.org/0000-0003-0136-4853Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing, ChinaKey Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing, ChinaKey Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing, ChinaKey Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing, ChinaKey Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing, ChinaFace sketch synthesis has made significant progress in the past few years. Recently, GAN-based methods have shown promising results on image-to-image translation problems, especially photo-to-sketch synthesis. Because the facial sketch has a hyper-abstract style and continuous graphic elements, compared with other image styles, its local details are easier to expose small artifacts and blur. The existing face sketch synthesis methods lack models for specific facial regions and usually generate face sketches with coarse structures. To synthesis high-quality sketches and overcome the blurs and deformations, this paper proposes a novel Multi-Hierarchies GAN, which divides the face image into multiple hierarchical structures to learn different regions' features of the face. It includes three modules: a local region module, mask module, and fusion module. The local region module can learn the detailed features of different local regions of the face by GAN. The mask module can generate a coarse facial structure of a sketch and uses the facial feature extractor to enhance the high-level image and learn the latent spaces' feature. The fusion module can generate the final sketch by combining fine local regions and coarse facial structure. Extensive qualitative and quantitative experiments illustrate that the proposed method outperforms the state-of-the-art methods on the CUFS and CUFSF standard datasets and photos on the internet.https://ieeexplore.ieee.org/document/9272961/Face sketch synthesisgenerative adversarial networkfacial feature extractormulti-hierarchies GAN |
| spellingShingle | Kangning Du Huaqiang Zhou Lin Cao Yanan Guo Tao Wang MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis IEEE Access Face sketch synthesis generative adversarial network facial feature extractor multi-hierarchies GAN |
| title | MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis |
| title_full | MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis |
| title_fullStr | MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis |
| title_full_unstemmed | MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis |
| title_short | MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis |
| title_sort | mhgan multi hierarchies generative adversarial network for high quality face sketch synthesis |
| topic | Face sketch synthesis generative adversarial network facial feature extractor multi-hierarchies GAN |
| url | https://ieeexplore.ieee.org/document/9272961/ |
| work_keys_str_mv | AT kangningdu mhganmultihierarchiesgenerativeadversarialnetworkforhighqualityfacesketchsynthesis AT huaqiangzhou mhganmultihierarchiesgenerativeadversarialnetworkforhighqualityfacesketchsynthesis AT lincao mhganmultihierarchiesgenerativeadversarialnetworkforhighqualityfacesketchsynthesis AT yananguo mhganmultihierarchiesgenerativeadversarialnetworkforhighqualityfacesketchsynthesis AT taowang mhganmultihierarchiesgenerativeadversarialnetworkforhighqualityfacesketchsynthesis |