Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory
Fast Style Transfer is a series of Neural Style Transfer algorithms that use feed-forward neural networks to render input images. Because of the high dimension of the output layer, these networks require much memory for computation. Therefore, for high-resolution images, most mobile devices and pers...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9179737/ |
| _version_ | 1818479407953608704 |
|---|---|
| author | Weifeng Ma Zhe Chen Caoting Ji |
| author_facet | Weifeng Ma Zhe Chen Caoting Ji |
| author_sort | Weifeng Ma |
| collection | DOAJ |
| description | Fast Style Transfer is a series of Neural Style Transfer algorithms that use feed-forward neural networks to render input images. Because of the high dimension of the output layer, these networks require much memory for computation. Therefore, for high-resolution images, most mobile devices and personal computers cannot stylize them, which greatly limits the application scenarios of Fast Style Transfer. At present, the two existing solutions are purchasing more memory and using the feathering-based method, but the former requires additional cost, and the latter has poor image quality. To solve this problem, we propose a novel image synthesis method named <italic>block shuffle</italic>, which converts a single task with high memory consumption to multiple subtasks with low memory consumption. This method can act as a plug-in for Fast Style Transfer without any modification to the network architecture. We use the most popular Fast Style Transfer repository on GitHub as the baseline. Experiments show that the quality of high-resolution images generated by our method is better than that of the feathering-based method. Although our method is an order of magnitude slower than the baseline, it can stylize high-resolution images with limited memory, which is impossible with the baseline. The code, models, and Android demonstration application will be made available on <uri>https://github.com/czczup/block-shuffle</uri>. |
| first_indexed | 2024-12-10T11:10:12Z |
| format | Article |
| id | doaj.art-f5bbc9dc8e544e5cb8b9dfe74a5f5e1f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T11:10:12Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-f5bbc9dc8e544e5cb8b9dfe74a5f5e1f2022-12-22T01:51:28ZengIEEEIEEE Access2169-35362020-01-01815805615806610.1109/ACCESS.2020.30200539179737Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited MemoryWeifeng Ma0https://orcid.org/0000-0002-7345-8907Zhe Chen1https://orcid.org/0000-0003-3419-8812Caoting Ji2https://orcid.org/0000-0002-4857-1035School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaSchool of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaSchool of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaFast Style Transfer is a series of Neural Style Transfer algorithms that use feed-forward neural networks to render input images. Because of the high dimension of the output layer, these networks require much memory for computation. Therefore, for high-resolution images, most mobile devices and personal computers cannot stylize them, which greatly limits the application scenarios of Fast Style Transfer. At present, the two existing solutions are purchasing more memory and using the feathering-based method, but the former requires additional cost, and the latter has poor image quality. To solve this problem, we propose a novel image synthesis method named <italic>block shuffle</italic>, which converts a single task with high memory consumption to multiple subtasks with low memory consumption. This method can act as a plug-in for Fast Style Transfer without any modification to the network architecture. We use the most popular Fast Style Transfer repository on GitHub as the baseline. Experiments show that the quality of high-resolution images generated by our method is better than that of the feathering-based method. Although our method is an order of magnitude slower than the baseline, it can stylize high-resolution images with limited memory, which is impossible with the baseline. The code, models, and Android demonstration application will be made available on <uri>https://github.com/czczup/block-shuffle</uri>.https://ieeexplore.ieee.org/document/9179737/Fast style transferhigh-resolutionlimited memory |
| spellingShingle | Weifeng Ma Zhe Chen Caoting Ji Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory IEEE Access Fast style transfer high-resolution limited memory |
| title | Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory |
| title_full | Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory |
| title_fullStr | Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory |
| title_full_unstemmed | Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory |
| title_short | Block Shuffle: A Method for High-Resolution Fast Style Transfer With Limited Memory |
| title_sort | block shuffle a method for high resolution fast style transfer with limited memory |
| topic | Fast style transfer high-resolution limited memory |
| url | https://ieeexplore.ieee.org/document/9179737/ |
| work_keys_str_mv | AT weifengma blockshuffleamethodforhighresolutionfaststyletransferwithlimitedmemory AT zhechen blockshuffleamethodforhighresolutionfaststyletransferwithlimitedmemory AT caotingji blockshuffleamethodforhighresolutionfaststyletransferwithlimitedmemory |