Refining Line Art From Stroke Style Disentanglement With Diffusion Models
A beginner who wants to create illustrations has difficulty improving his/her ability without expert advice. Especially in the initial steps, line drawings are critical but hard to evaluate because there are many assessment points, such as shape, variation in thickness, stroke fluency, and shadow ex...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10374353/ |
| _version_ | 1827376440744607744 |
|---|---|
| author | Fanglu Xie Motohiro Takagi Hitoshi Seshimo Yushi Aono |
| author_facet | Fanglu Xie Motohiro Takagi Hitoshi Seshimo Yushi Aono |
| author_sort | Fanglu Xie |
| collection | DOAJ |
| description | A beginner who wants to create illustrations has difficulty improving his/her ability without expert advice. Especially in the initial steps, line drawings are critical but hard to evaluate because there are many assessment points, such as shape, variation in thickness, stroke fluency, and shadow expression. Moreover, there is no well-summarized line art dataset based on expert knowledge to support skill refinement. Furthermore, the evaluation criterion is always subjective. To solve this problem, we custom-build systematized line artworks formed by cataloged stroke styles and propose a machine learning method that can automatically give clues to refining the artworks. We request 10 professional-level artists to create line art in six patterns; the stroke styles of the images are systematically summarized. Using this specific dataset, we train an auxiliary classifier to identify and remove features of those patterns to refine all line artwork commonly. We also implement an enhancement step that uses diffusion models to add more informative details to the generated results. The proposed method can automatically identify where strokes are needed to change and generate high-quality refined versions. Our method performs better than the previous method regarding L2, lpips, and SSIM scores while giving specialized clues to different stroke styles. |
| first_indexed | 2024-03-08T12:09:54Z |
| format | Article |
| id | doaj.art-ac3e33c45ae8430482a3e19ba2c7d8fd |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T12:09:54Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-ac3e33c45ae8430482a3e19ba2c7d8fd2024-01-23T00:02:39ZengIEEEIEEE Access2169-35362024-01-01129526953510.1109/ACCESS.2023.334755110374353Refining Line Art From Stroke Style Disentanglement With Diffusion ModelsFanglu Xie0https://orcid.org/0009-0009-3756-3599Motohiro Takagi1Hitoshi Seshimo2Yushi Aono3NTT Human Informatics Laboratories, Kanagawa, JapanNTT Human Informatics Laboratories, Kanagawa, JapanNTT Human Informatics Laboratories, Kanagawa, JapanNTT Human Informatics Laboratories, Kanagawa, JapanA beginner who wants to create illustrations has difficulty improving his/her ability without expert advice. Especially in the initial steps, line drawings are critical but hard to evaluate because there are many assessment points, such as shape, variation in thickness, stroke fluency, and shadow expression. Moreover, there is no well-summarized line art dataset based on expert knowledge to support skill refinement. Furthermore, the evaluation criterion is always subjective. To solve this problem, we custom-build systematized line artworks formed by cataloged stroke styles and propose a machine learning method that can automatically give clues to refining the artworks. We request 10 professional-level artists to create line art in six patterns; the stroke styles of the images are systematically summarized. Using this specific dataset, we train an auxiliary classifier to identify and remove features of those patterns to refine all line artwork commonly. We also implement an enhancement step that uses diffusion models to add more informative details to the generated results. The proposed method can automatically identify where strokes are needed to change and generate high-quality refined versions. Our method performs better than the previous method regarding L2, lpips, and SSIM scores while giving specialized clues to different stroke styles.https://ieeexplore.ieee.org/document/10374353/Disentangled representationimage generationline art refinementdenoising diffusion probabilistic models |
| spellingShingle | Fanglu Xie Motohiro Takagi Hitoshi Seshimo Yushi Aono Refining Line Art From Stroke Style Disentanglement With Diffusion Models IEEE Access Disentangled representation image generation line art refinement denoising diffusion probabilistic models |
| title | Refining Line Art From Stroke Style Disentanglement With Diffusion Models |
| title_full | Refining Line Art From Stroke Style Disentanglement With Diffusion Models |
| title_fullStr | Refining Line Art From Stroke Style Disentanglement With Diffusion Models |
| title_full_unstemmed | Refining Line Art From Stroke Style Disentanglement With Diffusion Models |
| title_short | Refining Line Art From Stroke Style Disentanglement With Diffusion Models |
| title_sort | refining line art from stroke style disentanglement with diffusion models |
| topic | Disentangled representation image generation line art refinement denoising diffusion probabilistic models |
| url | https://ieeexplore.ieee.org/document/10374353/ |
| work_keys_str_mv | AT fangluxie refininglineartfromstrokestyledisentanglementwithdiffusionmodels AT motohirotakagi refininglineartfromstrokestyledisentanglementwithdiffusionmodels AT hitoshiseshimo refininglineartfromstrokestyledisentanglementwithdiffusionmodels AT yushiaono refininglineartfromstrokestyledisentanglementwithdiffusionmodels |