AUPOD: End-to-End Automatic Poster Design by Self-Supervision
The automatic design has become a popular topic in the application field of computer vision technologies. Previous methods for automatic design are mostly saliency-based, relying on an off-the-shelf model for saliency map detection and hand-crafted aesthetic rules for ranking on multiple proposals....
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9764746/ |
| _version_ | 1817985836025643008 |
|---|---|
| author | Dongjin Huang Jinyao Li Chuanman Liu Jinhua Liu |
| author_facet | Dongjin Huang Jinyao Li Chuanman Liu Jinhua Liu |
| author_sort | Dongjin Huang |
| collection | DOAJ |
| description | The automatic design has become a popular topic in the application field of computer vision technologies. Previous methods for automatic design are mostly saliency-based, relying on an off-the-shelf model for saliency map detection and hand-crafted aesthetic rules for ranking on multiple proposals. We argue that the multi-stage generation and the excessive reliance on saliency map hindered the progress of pursuing better automatic design solutions. In this work, we explore the possibility of a saliency-free solution in a representative scenario, automatic poster design. We propose a novel end-to-end framework to solve the automatic poster design problem, which is divided into the layout prediction and attributes identification sub-tasks. We design a neural network based on multi-modality feature extraction to learn the two sub-tasks jointly. We train the deep neural network in our framework with automatically extracted supervision from semi-structured posters, bypassing a large amount of required manual labor. Both qualitative and quantitative results show the impressive performance of our end-to-end approach after discarding the explicit saliency detection module. Our system learned on self-supervision performs well on the automatic design by learning aesthetic constraints implicitly in the neural networks. |
| first_indexed | 2024-04-14T00:03:01Z |
| format | Article |
| id | doaj.art-5d80df7af7d746bba52d2723f067b5f7 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-14T00:03:01Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5d80df7af7d746bba52d2723f067b5f72022-12-22T02:23:38ZengIEEEIEEE Access2169-35362022-01-0110473484736010.1109/ACCESS.2022.31710339764746AUPOD: End-to-End Automatic Poster Design by Self-SupervisionDongjin Huang0Jinyao Li1https://orcid.org/0000-0003-3753-8907Chuanman Liu2Jinhua Liu3Shanghai Film Academy, Shanghai University, Shanghai, ChinaShanghai Film Academy, Shanghai University, Shanghai, ChinaShanghai Film Academy, Shanghai University, Shanghai, ChinaShanghai Film Academy, Shanghai University, Shanghai, ChinaThe automatic design has become a popular topic in the application field of computer vision technologies. Previous methods for automatic design are mostly saliency-based, relying on an off-the-shelf model for saliency map detection and hand-crafted aesthetic rules for ranking on multiple proposals. We argue that the multi-stage generation and the excessive reliance on saliency map hindered the progress of pursuing better automatic design solutions. In this work, we explore the possibility of a saliency-free solution in a representative scenario, automatic poster design. We propose a novel end-to-end framework to solve the automatic poster design problem, which is divided into the layout prediction and attributes identification sub-tasks. We design a neural network based on multi-modality feature extraction to learn the two sub-tasks jointly. We train the deep neural network in our framework with automatically extracted supervision from semi-structured posters, bypassing a large amount of required manual labor. Both qualitative and quantitative results show the impressive performance of our end-to-end approach after discarding the explicit saliency detection module. Our system learned on self-supervision performs well on the automatic design by learning aesthetic constraints implicitly in the neural networks.https://ieeexplore.ieee.org/document/9764746/Design automationdesign aestheticartificial intelligenceneural networksmachine learning |
| spellingShingle | Dongjin Huang Jinyao Li Chuanman Liu Jinhua Liu AUPOD: End-to-End Automatic Poster Design by Self-Supervision IEEE Access Design automation design aesthetic artificial intelligence neural networks machine learning |
| title | AUPOD: End-to-End Automatic Poster Design by Self-Supervision |
| title_full | AUPOD: End-to-End Automatic Poster Design by Self-Supervision |
| title_fullStr | AUPOD: End-to-End Automatic Poster Design by Self-Supervision |
| title_full_unstemmed | AUPOD: End-to-End Automatic Poster Design by Self-Supervision |
| title_short | AUPOD: End-to-End Automatic Poster Design by Self-Supervision |
| title_sort | aupod end to end automatic poster design by self supervision |
| topic | Design automation design aesthetic artificial intelligence neural networks machine learning |
| url | https://ieeexplore.ieee.org/document/9764746/ |
| work_keys_str_mv | AT dongjinhuang aupodendtoendautomaticposterdesignbyselfsupervision AT jinyaoli aupodendtoendautomaticposterdesignbyselfsupervision AT chuanmanliu aupodendtoendautomaticposterdesignbyselfsupervision AT jinhualiu aupodendtoendautomaticposterdesignbyselfsupervision |