An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches
An illustrative sketch style expresses important shapes and regions of objects and scenes with salient lines and dark tones, while abstracting less important shapes and regions as vacant spaces. We present a framework that produces illustrative sketch styles from various photographs. Our framework i...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/9/21/2791 |
| _version_ | 1797512054496559104 |
|---|---|
| author | Jihyeon Yeom Heekyung Yang Kyungha Min |
| author_facet | Jihyeon Yeom Heekyung Yang Kyungha Min |
| author_sort | Jihyeon Yeom |
| collection | DOAJ |
| description | An illustrative sketch style expresses important shapes and regions of objects and scenes with salient lines and dark tones, while abstracting less important shapes and regions as vacant spaces. We present a framework that produces illustrative sketch styles from various photographs. Our framework is designed using a generative adversarial network (GAN), which comprised four modules: a style extraction module, a generator module, a discriminator module and RCCL module. We devise two key ideas to effectively extract illustrative sketch styles from sample artworks and to apply them to input photographs. The first idea is using an attention map that extracts the required style features from important shapes and regions of sample illustrative sketch styles. This attention map is used in the generator module of our framework for the effective production of illustrative sketch styles. The second idea is using a relaxed cycle consistency loss that evaluates the quality of the produced illustrative sketch styles by comparing images that are reconstructed from the produced illustrative sketch styles and the input photographs. This relaxed cycle consistency loss focuses on the comparison of important shapes and regions for an effective evaluation of the quality of the produced illustrative sketch styles. Our GAN-based framework with an attention map and a relaxed cycle consistency loss effectively produces illustrative sketch styles on various target photographs, including portraits, landscapes, and still lifes. We demonstrate the effectiveness of our framework through a human study, ablation study, and Frechet Inception Distance evaluation. |
| first_indexed | 2024-03-10T05:56:29Z |
| format | Article |
| id | doaj.art-1073e24c3cd14971836f20cd34525cea |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-10T05:56:29Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-1073e24c3cd14971836f20cd34525cea2023-11-22T21:18:55ZengMDPI AGMathematics2227-73902021-11-01921279110.3390/math9212791An Attention-Based Generative Adversarial Network for Producing Illustrative SketchesJihyeon Yeom0Heekyung Yang1Kyungha Min2Department of Computer Science, Sangmyung University, Seoul 03016, KoreaDivision of SW Convergence, Sangmyung University, Seoul 03016, KoreaDepartment of Computer Science, Sangmyung University, Seoul 03016, KoreaAn illustrative sketch style expresses important shapes and regions of objects and scenes with salient lines and dark tones, while abstracting less important shapes and regions as vacant spaces. We present a framework that produces illustrative sketch styles from various photographs. Our framework is designed using a generative adversarial network (GAN), which comprised four modules: a style extraction module, a generator module, a discriminator module and RCCL module. We devise two key ideas to effectively extract illustrative sketch styles from sample artworks and to apply them to input photographs. The first idea is using an attention map that extracts the required style features from important shapes and regions of sample illustrative sketch styles. This attention map is used in the generator module of our framework for the effective production of illustrative sketch styles. The second idea is using a relaxed cycle consistency loss that evaluates the quality of the produced illustrative sketch styles by comparing images that are reconstructed from the produced illustrative sketch styles and the input photographs. This relaxed cycle consistency loss focuses on the comparison of important shapes and regions for an effective evaluation of the quality of the produced illustrative sketch styles. Our GAN-based framework with an attention map and a relaxed cycle consistency loss effectively produces illustrative sketch styles on various target photographs, including portraits, landscapes, and still lifes. We demonstrate the effectiveness of our framework through a human study, ablation study, and Frechet Inception Distance evaluation.https://www.mdpi.com/2227-7390/9/21/2791deep learningGANattention mapstylizationillustrative sketch |
| spellingShingle | Jihyeon Yeom Heekyung Yang Kyungha Min An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches Mathematics deep learning GAN attention map stylization illustrative sketch |
| title | An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches |
| title_full | An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches |
| title_fullStr | An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches |
| title_full_unstemmed | An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches |
| title_short | An Attention-Based Generative Adversarial Network for Producing Illustrative Sketches |
| title_sort | attention based generative adversarial network for producing illustrative sketches |
| topic | deep learning GAN attention map stylization illustrative sketch |
| url | https://www.mdpi.com/2227-7390/9/21/2791 |
| work_keys_str_mv | AT jihyeonyeom anattentionbasedgenerativeadversarialnetworkforproducingillustrativesketches AT heekyungyang anattentionbasedgenerativeadversarialnetworkforproducingillustrativesketches AT kyunghamin anattentionbasedgenerativeadversarialnetworkforproducingillustrativesketches AT jihyeonyeom attentionbasedgenerativeadversarialnetworkforproducingillustrativesketches AT heekyungyang attentionbasedgenerativeadversarialnetworkforproducingillustrativesketches AT kyunghamin attentionbasedgenerativeadversarialnetworkforproducingillustrativesketches |