Personality-assisted multi-task learning for generic and personalized image aesthetics assessment
Traditional image aesthetics assessment (IAA) approaches mainly predict the average aesthetic score of an image. However, people tend to have different tastes on image aesthetics, which is mainly determined by their subjective preferences. As an important subjective trait, personality is believed to...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/154486 |
| _version_ | 1826124018767364096 |
|---|---|
| author | Li, Leida Zhu, Hancheng Zhao, Sicheng Ding, Guiguang Lin, Weisi |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Li, Leida Zhu, Hancheng Zhao, Sicheng Ding, Guiguang Lin, Weisi |
| author_sort | Li, Leida |
| collection | NTU |
| description | Traditional image aesthetics assessment (IAA) approaches mainly predict the average aesthetic score of an image. However, people tend to have different tastes on image aesthetics, which is mainly determined by their subjective preferences. As an important subjective trait, personality is believed to be a key factor in modeling individual's subjective preference. In this paper, we present a personality-assisted multi-task deep learning framework for both generic and personalized image aesthetics assessment. The proposed framework comprises two stages. In the first stage, a multi-task learning network with shared weights is proposed to predict the aesthetics distribution of an image and Big-Five (BF) personality traits of people who like the image. The generic aesthetics score of the image can be generated based on the predicted aesthetics distribution. In order to capture the common representation of generic image aesthetics and people's personality traits, a Siamese network is trained using aesthetics data and personality data jointly. In the second stage, based on the predicted personality traits and generic aesthetics of an image, an inter-task fusion is introduced to generate individual's personalized aesthetic scores on the image. The performance of the proposed method is evaluated using two public image aesthetics databases. The experimental results demonstrate that the proposed method outperforms the state-of-the-arts in both generic and personalized IAA tasks. |
| first_indexed | 2024-10-01T06:13:48Z |
| format | Journal Article |
| id | ntu-10356/154486 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:13:48Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | ntu-10356/1544862021-12-23T06:52:57Z Personality-assisted multi-task learning for generic and personalized image aesthetics assessment Li, Leida Zhu, Hancheng Zhao, Sicheng Ding, Guiguang Lin, Weisi School of Computer Science and Engineering Engineering::Computer science and engineering Image Aesthetics Assessment Generic and Personalized Image Aesthetics Traditional image aesthetics assessment (IAA) approaches mainly predict the average aesthetic score of an image. However, people tend to have different tastes on image aesthetics, which is mainly determined by their subjective preferences. As an important subjective trait, personality is believed to be a key factor in modeling individual's subjective preference. In this paper, we present a personality-assisted multi-task deep learning framework for both generic and personalized image aesthetics assessment. The proposed framework comprises two stages. In the first stage, a multi-task learning network with shared weights is proposed to predict the aesthetics distribution of an image and Big-Five (BF) personality traits of people who like the image. The generic aesthetics score of the image can be generated based on the predicted aesthetics distribution. In order to capture the common representation of generic image aesthetics and people's personality traits, a Siamese network is trained using aesthetics data and personality data jointly. In the second stage, based on the predicted personality traits and generic aesthetics of an image, an inter-task fusion is introduced to generate individual's personalized aesthetic scores on the image. The performance of the proposed method is evaluated using two public image aesthetics databases. The experimental results demonstrate that the proposed method outperforms the state-of-the-arts in both generic and personalized IAA tasks. This work was supported in part by the Natural Science Foundation of Jiangsu Province under Grant BK20181354, in part by the Postgraduate Research & Practice Innovation Program of Jiangsu Province under Grant KYCX19_2142, in part by the Postgraduate Research & Practice Innovation Program of China, University of Mining and Technology, under Grant KYCX19_2142, in part by the National Natural Science Foundation of China under Grant 61701273, Grant 61771473, and Grant 61379143, in part by the Six Talent Peaks High-level Talents in Jiangsu Province under Grant XYDXX-063, and in part by the Qing Lan Project. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Joao M. Ascenso. 2021-12-23T06:52:57Z 2021-12-23T06:52:57Z 2020 Journal Article Li, L., Zhu, H., Zhao, S., Ding, G. & Lin, W. (2020). Personality-assisted multi-task learning for generic and personalized image aesthetics assessment. IEEE Transactions On Image Processing, 29, 3898-3910. https://dx.doi.org/10.1109/TIP.2020.2968285 1057-7149 https://hdl.handle.net/10356/154486 10.1109/TIP.2020.2968285 31995495 2-s2.0-85095425152 29 3898 3910 en IEEE Transactions On Image Processing © 2020 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Image Aesthetics Assessment Generic and Personalized Image Aesthetics Li, Leida Zhu, Hancheng Zhao, Sicheng Ding, Guiguang Lin, Weisi Personality-assisted multi-task learning for generic and personalized image aesthetics assessment |
| title | Personality-assisted multi-task learning for generic and personalized image aesthetics assessment |
| title_full | Personality-assisted multi-task learning for generic and personalized image aesthetics assessment |
| title_fullStr | Personality-assisted multi-task learning for generic and personalized image aesthetics assessment |
| title_full_unstemmed | Personality-assisted multi-task learning for generic and personalized image aesthetics assessment |
| title_short | Personality-assisted multi-task learning for generic and personalized image aesthetics assessment |
| title_sort | personality assisted multi task learning for generic and personalized image aesthetics assessment |
| topic | Engineering::Computer science and engineering Image Aesthetics Assessment Generic and Personalized Image Aesthetics |
| url | https://hdl.handle.net/10356/154486 |
| work_keys_str_mv | AT lileida personalityassistedmultitasklearningforgenericandpersonalizedimageaestheticsassessment AT zhuhancheng personalityassistedmultitasklearningforgenericandpersonalizedimageaestheticsassessment AT zhaosicheng personalityassistedmultitasklearningforgenericandpersonalizedimageaestheticsassessment AT dingguiguang personalityassistedmultitasklearningforgenericandpersonalizedimageaestheticsassessment AT linweisi personalityassistedmultitasklearningforgenericandpersonalizedimageaestheticsassessment |