Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media
In social networks, users can easily share information and express their opinions. Given the huge amount of data posted by many users, it is difficult to search for relevant information. In addition to individual posts, it would be useful if we can recommend groups of people with similar interests....
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/3/1064 |
| _version_ | 1797407792598876160 |
|---|---|
| author | Jenq-Haur Wang Yen-Tsang Wu Long Wang |
| author_facet | Jenq-Haur Wang Yen-Tsang Wu Long Wang |
| author_sort | Jenq-Haur Wang |
| collection | DOAJ |
| description | In social networks, users can easily share information and express their opinions. Given the huge amount of data posted by many users, it is difficult to search for relevant information. In addition to individual posts, it would be useful if we can recommend groups of people with similar interests. Past studies on user preference learning focused on single-modal features such as review contents or demographic information of users. However, such information is usually not easy to obtain in most social media without explicit user feedback. In this paper, we propose a multimodal feature fusion approach to implicit user preference prediction which combines text and image features from user posts for recommending similar users in social media. First, we use the convolutional neural network (CNN) and TextCNN models to extract image and text features, respectively. Then, these features are combined using early and late fusion methods as a representation of user preferences. Lastly, a list of users with the most similar preferences are recommended. The experimental results on real-world Instagram data show that the best performance can be achieved when we apply late fusion of individual classification results for images and texts, with the best average top-k accuracy of 0.491. This validates the effectiveness of utilizing deep learning methods for fusing multimodal features to represent social user preferences. Further investigation is needed to verify the performance in different types of social media. |
| first_indexed | 2024-03-09T03:46:46Z |
| format | Article |
| id | doaj.art-c868c17fd9bc40f0970b4e11df602a13 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T03:46:46Z |
| publishDate | 2021-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-c868c17fd9bc40f0970b4e11df602a132023-12-03T14:33:41ZengMDPI AGApplied Sciences2076-34172021-01-01113106410.3390/app11031064Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social MediaJenq-Haur Wang0Yen-Tsang Wu1Long Wang2Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 106344, TaiwanDepartment of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 106344, TaiwanSchool of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, ChinaIn social networks, users can easily share information and express their opinions. Given the huge amount of data posted by many users, it is difficult to search for relevant information. In addition to individual posts, it would be useful if we can recommend groups of people with similar interests. Past studies on user preference learning focused on single-modal features such as review contents or demographic information of users. However, such information is usually not easy to obtain in most social media without explicit user feedback. In this paper, we propose a multimodal feature fusion approach to implicit user preference prediction which combines text and image features from user posts for recommending similar users in social media. First, we use the convolutional neural network (CNN) and TextCNN models to extract image and text features, respectively. Then, these features are combined using early and late fusion methods as a representation of user preferences. Lastly, a list of users with the most similar preferences are recommended. The experimental results on real-world Instagram data show that the best performance can be achieved when we apply late fusion of individual classification results for images and texts, with the best average top-k accuracy of 0.491. This validates the effectiveness of utilizing deep learning methods for fusing multimodal features to represent social user preferences. Further investigation is needed to verify the performance in different types of social media.https://www.mdpi.com/2076-3417/11/3/1064deep learninguser preference learningfeature fusionsimilar user recommendation |
| spellingShingle | Jenq-Haur Wang Yen-Tsang Wu Long Wang Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media Applied Sciences deep learning user preference learning feature fusion similar user recommendation |
| title | Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media |
| title_full | Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media |
| title_fullStr | Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media |
| title_full_unstemmed | Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media |
| title_short | Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media |
| title_sort | predicting implicit user preferences with multimodal feature fusion for similar user recommendation in social media |
| topic | deep learning user preference learning feature fusion similar user recommendation |
| url | https://www.mdpi.com/2076-3417/11/3/1064 |
| work_keys_str_mv | AT jenqhaurwang predictingimplicituserpreferenceswithmultimodalfeaturefusionforsimilaruserrecommendationinsocialmedia AT yentsangwu predictingimplicituserpreferenceswithmultimodalfeaturefusionforsimilaruserrecommendationinsocialmedia AT longwang predictingimplicituserpreferenceswithmultimodalfeaturefusionforsimilaruserrecommendationinsocialmedia |