Tag-based image retrieval improved by augmented features and group-based refinement
In this paper, we propose a new tag-based image retrieval framework to improve the retrieval performance of a group of related personal images captured by the same user within a short period of an event by leveraging millions of training web images and their associated rich textual descriptions. For...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2013
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/96254 http://hdl.handle.net/10220/11473 |
| _version_ | 1826126611752157184 |
|---|---|
| author | Chen, Lin Xu, Dong Tsang, Ivor Wai-Hung Luo, Jiebo |
| author2 | School of Computer Engineering |
| author_facet | School of Computer Engineering Chen, Lin Xu, Dong Tsang, Ivor Wai-Hung Luo, Jiebo |
| author_sort | Chen, Lin |
| collection | NTU |
| description | In this paper, we propose a new tag-based image retrieval framework to improve the retrieval performance of a group of related personal images captured by the same user within a short period of an event by leveraging millions of training web images and their associated rich textual descriptions. For any given query tag (e.g., “car”), the inverted file method is employed to automatically determine the relevant training web images that are associated with the query tag and the irrelevant training web images that are not associated with the query tag. Using these relevant and irrelevant web images as positive and negative training data respectively, we propose a new classification method called support vector machine (SVM) with augmented features (AFSVM) to learn an adapted classifier by leveraging the prelearned SVM classifiers of popular tags that are associated with a large number of relevant training web images. Treating the decision values of one group of test photos from AFSVM classifiers as the initial relevance scores, in the subsequent group-based refinement process, we propose to use the Laplacian regularized least squares method to further refine the relevance scores of test photos by utilizing the visual similarity of the images within the group. Based on the refined relevance scores, our proposed framework can be readily applied to tag-based image retrieval for a group of raw consumer photos without any textual descriptions or a group of Flickr photos with noisy tags. Moreover, we propose a new method to better calculate the relevance scores for Flickr photos. Extensive experiments on two datasets demonstrate the effectiveness of our framework. |
| first_indexed | 2024-10-01T06:55:29Z |
| format | Journal Article |
| id | ntu-10356/96254 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:55:29Z |
| publishDate | 2013 |
| record_format | dspace |
| spelling | ntu-10356/962542020-05-28T07:19:22Z Tag-based image retrieval improved by augmented features and group-based refinement Chen, Lin Xu, Dong Tsang, Ivor Wai-Hung Luo, Jiebo School of Computer Engineering Centre for Multimedia and Network Technology DRNTU::Engineering::Computer science and engineering In this paper, we propose a new tag-based image retrieval framework to improve the retrieval performance of a group of related personal images captured by the same user within a short period of an event by leveraging millions of training web images and their associated rich textual descriptions. For any given query tag (e.g., “car”), the inverted file method is employed to automatically determine the relevant training web images that are associated with the query tag and the irrelevant training web images that are not associated with the query tag. Using these relevant and irrelevant web images as positive and negative training data respectively, we propose a new classification method called support vector machine (SVM) with augmented features (AFSVM) to learn an adapted classifier by leveraging the prelearned SVM classifiers of popular tags that are associated with a large number of relevant training web images. Treating the decision values of one group of test photos from AFSVM classifiers as the initial relevance scores, in the subsequent group-based refinement process, we propose to use the Laplacian regularized least squares method to further refine the relevance scores of test photos by utilizing the visual similarity of the images within the group. Based on the refined relevance scores, our proposed framework can be readily applied to tag-based image retrieval for a group of raw consumer photos without any textual descriptions or a group of Flickr photos with noisy tags. Moreover, we propose a new method to better calculate the relevance scores for Flickr photos. Extensive experiments on two datasets demonstrate the effectiveness of our framework. 2013-07-15T08:50:15Z 2019-12-06T19:27:54Z 2013-07-15T08:50:15Z 2019-12-06T19:27:54Z 2012 2012 Journal Article Chen, L., Xu, D., Tsang, I. W., & Luo, J. (2012). Tag-Based Image Retrieval Improved by Augmented Features and Group-Based Refinement. IEEE Transactions on Multimedia, 14(4), 1057-1067. 1520-9210 https://hdl.handle.net/10356/96254 http://hdl.handle.net/10220/11473 10.1109/TMM.2012.2187435 en IEEE transactions on multimedia © 2012 IEEE. |
| spellingShingle | DRNTU::Engineering::Computer science and engineering Chen, Lin Xu, Dong Tsang, Ivor Wai-Hung Luo, Jiebo Tag-based image retrieval improved by augmented features and group-based refinement |
| title | Tag-based image retrieval improved by augmented features and group-based refinement |
| title_full | Tag-based image retrieval improved by augmented features and group-based refinement |
| title_fullStr | Tag-based image retrieval improved by augmented features and group-based refinement |
| title_full_unstemmed | Tag-based image retrieval improved by augmented features and group-based refinement |
| title_short | Tag-based image retrieval improved by augmented features and group-based refinement |
| title_sort | tag based image retrieval improved by augmented features and group based refinement |
| topic | DRNTU::Engineering::Computer science and engineering |
| url | https://hdl.handle.net/10356/96254 http://hdl.handle.net/10220/11473 |
| work_keys_str_mv | AT chenlin tagbasedimageretrievalimprovedbyaugmentedfeaturesandgroupbasedrefinement AT xudong tagbasedimageretrievalimprovedbyaugmentedfeaturesandgroupbasedrefinement AT tsangivorwaihung tagbasedimageretrievalimprovedbyaugmentedfeaturesandgroupbasedrefinement AT luojiebo tagbasedimageretrievalimprovedbyaugmentedfeaturesandgroupbasedrefinement |