Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module.
Recently, contrastive learning has gained popularity in the field of unsupervised image-to-image (I2I) translation. In a previous study, a query-selected attention (QS-Attn) module, which employed an attention matrix with a probability distribution, was used to maximize the mutual information betwee...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0293885&type=printable |
| _version_ | 1797628831327059968 |
|---|---|
| author | Yunseok Oh Seonhye Oh Sangwoo Noh Hangyu Kim Hyeon Seo |
| author_facet | Yunseok Oh Seonhye Oh Sangwoo Noh Hangyu Kim Hyeon Seo |
| author_sort | Yunseok Oh |
| collection | DOAJ |
| description | Recently, contrastive learning has gained popularity in the field of unsupervised image-to-image (I2I) translation. In a previous study, a query-selected attention (QS-Attn) module, which employed an attention matrix with a probability distribution, was used to maximize the mutual information between the source and translated images. This module selected significant queries using an entropy metric computed from the attention matrix. However, it often selected many queries with equal significance measures, leading to an excessive focus on the background. In this study, we proposed a dual-learning framework with QS-Attn and convolutional block attention module (CBAM) called object-stable dual contrastive learning generative adversarial network (OS-DCLGAN). In this paper, we utilize a CBAM, which learns what and where to emphasize or suppress, thereby refining intermediate features effectively. This CBAM was integrated before the QS-Attn module to capture significant domain information for I2I translation tasks. The proposed framework outperformed recently introduced approaches in various I2I translation tasks, showing its effectiveness and versatility. The code is available at https://github.com/RedPotatoChip/OSUDL. |
| first_indexed | 2024-03-11T10:45:44Z |
| format | Article |
| id | doaj.art-2fb7e7dd180f4601bfe2a7ea0d6e9f36 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-03-11T10:45:44Z |
| publishDate | 2023-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-2fb7e7dd180f4601bfe2a7ea0d6e9f362023-11-14T05:34:38ZengPublic Library of Science (PLoS)PLoS ONE1932-62032023-01-011811e029388510.1371/journal.pone.0293885Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module.Yunseok OhSeonhye OhSangwoo NohHangyu KimHyeon SeoRecently, contrastive learning has gained popularity in the field of unsupervised image-to-image (I2I) translation. In a previous study, a query-selected attention (QS-Attn) module, which employed an attention matrix with a probability distribution, was used to maximize the mutual information between the source and translated images. This module selected significant queries using an entropy metric computed from the attention matrix. However, it often selected many queries with equal significance measures, leading to an excessive focus on the background. In this study, we proposed a dual-learning framework with QS-Attn and convolutional block attention module (CBAM) called object-stable dual contrastive learning generative adversarial network (OS-DCLGAN). In this paper, we utilize a CBAM, which learns what and where to emphasize or suppress, thereby refining intermediate features effectively. This CBAM was integrated before the QS-Attn module to capture significant domain information for I2I translation tasks. The proposed framework outperformed recently introduced approaches in various I2I translation tasks, showing its effectiveness and versatility. The code is available at https://github.com/RedPotatoChip/OSUDL.https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0293885&type=printable |
| spellingShingle | Yunseok Oh Seonhye Oh Sangwoo Noh Hangyu Kim Hyeon Seo Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module. PLoS ONE |
| title | Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module. |
| title_full | Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module. |
| title_fullStr | Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module. |
| title_full_unstemmed | Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module. |
| title_short | Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module. |
| title_sort | object stable unsupervised dual contrastive learning image to image translation with query selected attention and convolutional block attention module |
| url | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0293885&type=printable |
| work_keys_str_mv | AT yunseokoh objectstableunsuperviseddualcontrastivelearningimagetoimagetranslationwithqueryselectedattentionandconvolutionalblockattentionmodule AT seonhyeoh objectstableunsuperviseddualcontrastivelearningimagetoimagetranslationwithqueryselectedattentionandconvolutionalblockattentionmodule AT sangwoonoh objectstableunsuperviseddualcontrastivelearningimagetoimagetranslationwithqueryselectedattentionandconvolutionalblockattentionmodule AT hangyukim objectstableunsuperviseddualcontrastivelearningimagetoimagetranslationwithqueryselectedattentionandconvolutionalblockattentionmodule AT hyeonseo objectstableunsuperviseddualcontrastivelearningimagetoimagetranslationwithqueryselectedattentionandconvolutionalblockattentionmodule |