LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer
To improve the generation quality of image style transfer, this paper proposes a light progressive attention adaptive instance normalization (LPAdaIN) model that combines the adaptive instance normalization (<i>AdaIN</i>) layer and the convolutional block attention module (CBAM). In the...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/11/18/2929 |
| _version_ | 1797489030809518080 |
|---|---|
| author | Qing Zhu Huang Bai Junmei Sun Chen Cheng Xiumei Li |
| author_facet | Qing Zhu Huang Bai Junmei Sun Chen Cheng Xiumei Li |
| author_sort | Qing Zhu |
| collection | DOAJ |
| description | To improve the generation quality of image style transfer, this paper proposes a light progressive attention adaptive instance normalization (LPAdaIN) model that combines the adaptive instance normalization (<i>AdaIN</i>) layer and the convolutional block attention module (CBAM). In the construction of the model structure, first, a lightweight autoencoder is built to reduce the information loss in the encoding process by reducing the number of network layers and to alleviate the distortion of the stylized image structure. Second, each <i>AdaIN</i> layer is progressively applied after the three relu layers in the encoder to obtain the fine-grained stylized feature maps. Third, the CBAM is added between the last <i>AdaIN</i> layer and the decoder, ensuring that the main objects in the stylized image are clearly visible. In the model optimization, a reconstruction loss is designed to improve the decoder’s ability to decode stylized images with more precise constraints and refine the structure of the stylized images. Compared with five classical style transfer models, the LPAdaIN is visually shown to more finely apply the texture of the style image to the content image, in order to obtain a stylized image, in which the main objects are clearly visible and the structure can be maintained. In terms of quantitative metrics, the LPAdaIN achieved good results in running speed and structural similarity. |
| first_indexed | 2024-03-10T00:10:43Z |
| format | Article |
| id | doaj.art-0125cd82d32d4419b2849f8d7c4f6b4d |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T00:10:43Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-0125cd82d32d4419b2849f8d7c4f6b4d2023-11-23T15:58:59ZengMDPI AGElectronics2079-92922022-09-011118292910.3390/electronics11182929LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style TransferQing Zhu0Huang Bai1Junmei Sun2Chen Cheng3Xiumei Li4School of Information Science and Technology, Hangzhou Normal University, Hangzhou 311121, ChinaSchool of Information Science and Technology, Hangzhou Normal University, Hangzhou 311121, ChinaSchool of Information Science and Technology, Hangzhou Normal University, Hangzhou 311121, ChinaSchool of Information Science and Technology, Hangzhou Normal University, Hangzhou 311121, ChinaSchool of Information Science and Technology, Hangzhou Normal University, Hangzhou 311121, ChinaTo improve the generation quality of image style transfer, this paper proposes a light progressive attention adaptive instance normalization (LPAdaIN) model that combines the adaptive instance normalization (<i>AdaIN</i>) layer and the convolutional block attention module (CBAM). In the construction of the model structure, first, a lightweight autoencoder is built to reduce the information loss in the encoding process by reducing the number of network layers and to alleviate the distortion of the stylized image structure. Second, each <i>AdaIN</i> layer is progressively applied after the three relu layers in the encoder to obtain the fine-grained stylized feature maps. Third, the CBAM is added between the last <i>AdaIN</i> layer and the decoder, ensuring that the main objects in the stylized image are clearly visible. In the model optimization, a reconstruction loss is designed to improve the decoder’s ability to decode stylized images with more precise constraints and refine the structure of the stylized images. Compared with five classical style transfer models, the LPAdaIN is visually shown to more finely apply the texture of the style image to the content image, in order to obtain a stylized image, in which the main objects are clearly visible and the structure can be maintained. In terms of quantitative metrics, the LPAdaIN achieved good results in running speed and structural similarity.https://www.mdpi.com/2079-9292/11/18/2929style transferlightweight autoencoderattention adaptive instance normalizationconvolutional block attention modelreconstruction loss |
| spellingShingle | Qing Zhu Huang Bai Junmei Sun Chen Cheng Xiumei Li LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer Electronics style transfer lightweight autoencoder attention adaptive instance normalization convolutional block attention model reconstruction loss |
| title | LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer |
| title_full | LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer |
| title_fullStr | LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer |
| title_full_unstemmed | LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer |
| title_short | LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer |
| title_sort | lpadain light progressive attention adaptive instance normalization model for style transfer |
| topic | style transfer lightweight autoencoder attention adaptive instance normalization convolutional block attention model reconstruction loss |
| url | https://www.mdpi.com/2079-9292/11/18/2929 |
| work_keys_str_mv | AT qingzhu lpadainlightprogressiveattentionadaptiveinstancenormalizationmodelforstyletransfer AT huangbai lpadainlightprogressiveattentionadaptiveinstancenormalizationmodelforstyletransfer AT junmeisun lpadainlightprogressiveattentionadaptiveinstancenormalizationmodelforstyletransfer AT chencheng lpadainlightprogressiveattentionadaptiveinstancenormalizationmodelforstyletransfer AT xiumeili lpadainlightprogressiveattentionadaptiveinstancenormalizationmodelforstyletransfer |