Local and blobal GANs with semantic-aware upsampling for image generation
In this paper, we address the task of semantic-guided image generation. One challenge common to most existing image-level generation methods is difficulty in generating small objects and detailed local textures. To tackle this issue, in this work we consider generating images using local context. As...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
IEEE
2022
|
| _version_ | 1826309651536281600 |
|---|---|
| author | Tang, H Shao, L Torr, P Sebe, N |
| author_facet | Tang, H Shao, L Torr, P Sebe, N |
| author_sort | Tang, H |
| collection | OXFORD |
| description | In this paper, we address the task of semantic-guided image generation. One challenge common to most existing image-level generation methods is difficulty in generating small objects and detailed local textures. To tackle this issue, in this work we consider generating images using local context. As such, we design a local class-specific generative network using semantic maps as guidance, which separately constructs and learns subgenerators for different classes, enabling it to capture finer details. To learn more discriminative class-specific feature representations for the local generation, we also propose a novel classification module. To combine the advantages of both global image-level and local class-specific generation, a joint generation network is designed with an attention fusion module and a dual-discriminator structure embedded. Lastly, we propose a novel semantic-aware upsampling method, which has a larger receptive field and can take far-away pixels that are semantically related for feature upsampling, enabling it to better preserve semantic consistency for instances with the same semantic labels. Extensive experiments on two image generation tasks show the superior performance of the proposed method. State-of-the-art results are established by large margins on both tasks and on nine challenging public benchmarks. |
| first_indexed | 2024-03-07T07:38:56Z |
| format | Journal article |
| id | oxford-uuid:1b479776-43ec-4c50-b4b3-11f58c5abacf |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:38:56Z |
| publishDate | 2022 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:1b479776-43ec-4c50-b4b3-11f58c5abacf2023-04-12T11:54:12ZLocal and blobal GANs with semantic-aware upsampling for image generationJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:1b479776-43ec-4c50-b4b3-11f58c5abacfEnglishSymplectic ElementsIEEE2022Tang, HShao, LTorr, PSebe, NIn this paper, we address the task of semantic-guided image generation. One challenge common to most existing image-level generation methods is difficulty in generating small objects and detailed local textures. To tackle this issue, in this work we consider generating images using local context. As such, we design a local class-specific generative network using semantic maps as guidance, which separately constructs and learns subgenerators for different classes, enabling it to capture finer details. To learn more discriminative class-specific feature representations for the local generation, we also propose a novel classification module. To combine the advantages of both global image-level and local class-specific generation, a joint generation network is designed with an attention fusion module and a dual-discriminator structure embedded. Lastly, we propose a novel semantic-aware upsampling method, which has a larger receptive field and can take far-away pixels that are semantically related for feature upsampling, enabling it to better preserve semantic consistency for instances with the same semantic labels. Extensive experiments on two image generation tasks show the superior performance of the proposed method. State-of-the-art results are established by large margins on both tasks and on nine challenging public benchmarks. |
| spellingShingle | Tang, H Shao, L Torr, P Sebe, N Local and blobal GANs with semantic-aware upsampling for image generation |
| title | Local and blobal GANs with semantic-aware upsampling for image generation |
| title_full | Local and blobal GANs with semantic-aware upsampling for image generation |
| title_fullStr | Local and blobal GANs with semantic-aware upsampling for image generation |
| title_full_unstemmed | Local and blobal GANs with semantic-aware upsampling for image generation |
| title_short | Local and blobal GANs with semantic-aware upsampling for image generation |
| title_sort | local and blobal gans with semantic aware upsampling for image generation |
| work_keys_str_mv | AT tangh localandblobalganswithsemanticawareupsamplingforimagegeneration AT shaol localandblobalganswithsemanticawareupsamplingforimagegeneration AT torrp localandblobalganswithsemanticawareupsamplingforimagegeneration AT seben localandblobalganswithsemanticawareupsamplingforimagegeneration |