Pseudo-Supervised Learning for Semantic Multi-Style Transfer
Numerous methods for style transfer have been developed using unsupervised learning and gained impressive results. However, optimal style transfer cannot be conducted from a global fashion in certain style domains, mainly when a single target-style domain contains semantic objects that have their ow...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9316188/ |
| _version_ | 1818479881764208640 |
|---|---|
| author | Saehun Kim Jeonghyeok Do Munchurl Kim |
| author_facet | Saehun Kim Jeonghyeok Do Munchurl Kim |
| author_sort | Saehun Kim |
| collection | DOAJ |
| description | Numerous methods for style transfer have been developed using unsupervised learning and gained impressive results. However, optimal style transfer cannot be conducted from a global fashion in certain style domains, mainly when a single target-style domain contains semantic objects that have their own distinct and unique styles, e.g., those objects in the anime-style domain. Previous methods are incongruent because the unsupervised learning can not provide the semantic mappings between the multi-style objects according to their unique styles. Thus, in this paper, we propose a pseudo-supervised learning framework for the semantic multi-style transfer (SMST), which consists of (i) a pseudo ground truth (pGT) generation phase and (ii) a SMST learning phase. In the pGT generation phase, multiple semantic objects of the photo images are separately transferred to the target-domain object styles in an object-oriented fashion. Then the transferred objects are composed back to an image, which is the pGT. In the SMST learning phase, a SMST network (SMSTnet) is trained with the pairs of the photo images and its respective pGT in a supervised manner. From this, our framework can provide the semantic mappings of multi-style objects. Moreover, to embrace the multi-styles of various objects into a single generator, we design the SMSTnet with channel attentions in conjunction with a discriminator dedicated to our pseudo-supervised learning. Our method has been applied and intensively tested for anime-style transfer learning. The experimental results demonstrate the effectiveness of our method and show its superiority compared to the state-of-the-art methods. |
| first_indexed | 2024-12-10T11:15:58Z |
| format | Article |
| id | doaj.art-03dbd0bc6f394002b5e8e3e924d22bf6 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T11:15:58Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-03dbd0bc6f394002b5e8e3e924d22bf62022-12-22T01:51:10ZengIEEEIEEE Access2169-35362021-01-0197930794210.1109/ACCESS.2021.30496379316188Pseudo-Supervised Learning for Semantic Multi-Style TransferSaehun Kim0https://orcid.org/0000-0002-7250-2654Jeonghyeok Do1https://orcid.org/0000-0003-0030-0129Munchurl Kim2https://orcid.org/0000-0003-0146-5419School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaNumerous methods for style transfer have been developed using unsupervised learning and gained impressive results. However, optimal style transfer cannot be conducted from a global fashion in certain style domains, mainly when a single target-style domain contains semantic objects that have their own distinct and unique styles, e.g., those objects in the anime-style domain. Previous methods are incongruent because the unsupervised learning can not provide the semantic mappings between the multi-style objects according to their unique styles. Thus, in this paper, we propose a pseudo-supervised learning framework for the semantic multi-style transfer (SMST), which consists of (i) a pseudo ground truth (pGT) generation phase and (ii) a SMST learning phase. In the pGT generation phase, multiple semantic objects of the photo images are separately transferred to the target-domain object styles in an object-oriented fashion. Then the transferred objects are composed back to an image, which is the pGT. In the SMST learning phase, a SMST network (SMSTnet) is trained with the pairs of the photo images and its respective pGT in a supervised manner. From this, our framework can provide the semantic mappings of multi-style objects. Moreover, to embrace the multi-styles of various objects into a single generator, we design the SMSTnet with channel attentions in conjunction with a discriminator dedicated to our pseudo-supervised learning. Our method has been applied and intensively tested for anime-style transfer learning. The experimental results demonstrate the effectiveness of our method and show its superiority compared to the state-of-the-art methods.https://ieeexplore.ieee.org/document/9316188/Style transferimage-to-image translationgenerative adversarial networks |
| spellingShingle | Saehun Kim Jeonghyeok Do Munchurl Kim Pseudo-Supervised Learning for Semantic Multi-Style Transfer IEEE Access Style transfer image-to-image translation generative adversarial networks |
| title | Pseudo-Supervised Learning for Semantic Multi-Style Transfer |
| title_full | Pseudo-Supervised Learning for Semantic Multi-Style Transfer |
| title_fullStr | Pseudo-Supervised Learning for Semantic Multi-Style Transfer |
| title_full_unstemmed | Pseudo-Supervised Learning for Semantic Multi-Style Transfer |
| title_short | Pseudo-Supervised Learning for Semantic Multi-Style Transfer |
| title_sort | pseudo supervised learning for semantic multi style transfer |
| topic | Style transfer image-to-image translation generative adversarial networks |
| url | https://ieeexplore.ieee.org/document/9316188/ |
| work_keys_str_mv | AT saehunkim pseudosupervisedlearningforsemanticmultistyletransfer AT jeonghyeokdo pseudosupervisedlearningforsemanticmultistyletransfer AT munchurlkim pseudosupervisedlearningforsemanticmultistyletransfer |