Texture Mixing by Interpolating Deep Statistics via Gaussian Models
Recently, enthusiastic studies have devoted to texture synthesis using deep neural networks, because these networks excel at handling complex patterns in images. In these models, second-order statistics, such as Gram matrix, are used to describe textures. Although these models have achieved promisin...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9052476/ |
| _version_ | 1818444199159136256 |
|---|---|
| author | Zhucun Xue Ziming Wang |
| author_facet | Zhucun Xue Ziming Wang |
| author_sort | Zhucun Xue |
| collection | DOAJ |
| description | Recently, enthusiastic studies have devoted to texture synthesis using deep neural networks, because these networks excel at handling complex patterns in images. In these models, second-order statistics, such as Gram matrix, are used to describe textures. Although these models have achieved promising results, the structure of their parametric space is still unclear. Consequently, it is difficult to use them to mix textures. This paper addresses the texture mixing problem by using a Gaussian scheme to interpolate deep statistics computed from deep neural networks. More precisely, we first reveal that the statistics used in existing deep models can be unified using a stationary Gaussian scheme. We then present a novel algorithm to mix these statistics by interpolating between Gaussian models using optimal transport. We further apply our scheme to Neural Style Transfer, where we can create mixed styles. The experiments demonstrate that our method outperforms a number of baselines. Because all the computations are implemented in closed forms, our mixing algorithm adds only negligible time to the original texture synthesis procedure. |
| first_indexed | 2024-12-14T19:12:08Z |
| format | Article |
| id | doaj.art-207c1f48dce54da0bd8a7654978b3794 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T19:12:08Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-207c1f48dce54da0bd8a7654978b37942022-12-21T22:50:42ZengIEEEIEEE Access2169-35362020-01-018607476075810.1109/ACCESS.2020.29814109052476Texture Mixing by Interpolating Deep Statistics via Gaussian ModelsZhucun Xue0Ziming Wang1https://orcid.org/0000-0002-1739-307XState Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, ChinaRecently, enthusiastic studies have devoted to texture synthesis using deep neural networks, because these networks excel at handling complex patterns in images. In these models, second-order statistics, such as Gram matrix, are used to describe textures. Although these models have achieved promising results, the structure of their parametric space is still unclear. Consequently, it is difficult to use them to mix textures. This paper addresses the texture mixing problem by using a Gaussian scheme to interpolate deep statistics computed from deep neural networks. More precisely, we first reveal that the statistics used in existing deep models can be unified using a stationary Gaussian scheme. We then present a novel algorithm to mix these statistics by interpolating between Gaussian models using optimal transport. We further apply our scheme to Neural Style Transfer, where we can create mixed styles. The experiments demonstrate that our method outperforms a number of baselines. Because all the computations are implemented in closed forms, our mixing algorithm adds only negligible time to the original texture synthesis procedure.https://ieeexplore.ieee.org/document/9052476/Texture modelingtexture mixingGaussian modelsdeep neural networks |
| spellingShingle | Zhucun Xue Ziming Wang Texture Mixing by Interpolating Deep Statistics via Gaussian Models IEEE Access Texture modeling texture mixing Gaussian models deep neural networks |
| title | Texture Mixing by Interpolating Deep Statistics via Gaussian Models |
| title_full | Texture Mixing by Interpolating Deep Statistics via Gaussian Models |
| title_fullStr | Texture Mixing by Interpolating Deep Statistics via Gaussian Models |
| title_full_unstemmed | Texture Mixing by Interpolating Deep Statistics via Gaussian Models |
| title_short | Texture Mixing by Interpolating Deep Statistics via Gaussian Models |
| title_sort | texture mixing by interpolating deep statistics via gaussian models |
| topic | Texture modeling texture mixing Gaussian models deep neural networks |
| url | https://ieeexplore.ieee.org/document/9052476/ |
| work_keys_str_mv | AT zhucunxue texturemixingbyinterpolatingdeepstatisticsviagaussianmodels AT zimingwang texturemixingbyinterpolatingdeepstatisticsviagaussianmodels |