Multi-granularity semantic information integration graph for cross-modal hash retrieval
With the development of intelligent collection technology and popularization of intelligent terminals, multi-source heterogeneous data are growing rapidly. The effective utilization of rich semantic information contained in massive amounts of multi-source heterogeneous data to provide users with hig...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers
2024
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/112890/1/112890.pdf |
| _version_ | 1825940215345184768 |
|---|---|
| author | Han, Zhichao Azman, Azreen Khalid, Fatimah Mustaffa, Mas Rina |
| author_facet | Han, Zhichao Azman, Azreen Khalid, Fatimah Mustaffa, Mas Rina |
| author_sort | Han, Zhichao |
| collection | UPM |
| description | With the development of intelligent collection technology and popularization of intelligent terminals, multi-source heterogeneous data are growing rapidly. The effective utilization of rich semantic information contained in massive amounts of multi-source heterogeneous data to provide users with high-quality cross-modal information retrieval services has become an urgent problem to be solved in the current field of information retrieval. In this paper, we propose a novel cross-modal retrieval method, named MGSGH, which deeply explores the internal correlation between data of different granularities by integrating coarse-grained global semantic information and fine-grained scene graph information to model global semantic concepts and local semantic relationship graphs within a modality respectively. By enforcing cross-modal consistency constraints and intra-modal similarity preservation, we effectively integrate the visual features of image data and semantic information of text data to overcome the heterogeneity between the two types of data. Furthermore, we propose a new method for learning hash codes directly, thereby reducing the impact of quantization loss. Our comprehensive experimental evaluation demonstrated the effectiveness and superiority of the proposed model in achieving accurate and efficient cross-modal retrieval. © 2013 IEEE. |
| first_indexed | 2024-12-09T02:24:49Z |
| format | Article |
| id | upm.eprints-112890 |
| institution | Universiti Putra Malaysia |
| language | English |
| last_indexed | 2024-12-09T02:24:49Z |
| publishDate | 2024 |
| publisher | Institute of Electrical and Electronics Engineers |
| record_format | dspace |
| spelling | upm.eprints-1128902024-10-28T08:11:55Z http://psasir.upm.edu.my/id/eprint/112890/ Multi-granularity semantic information integration graph for cross-modal hash retrieval Han, Zhichao Azman, Azreen Khalid, Fatimah Mustaffa, Mas Rina With the development of intelligent collection technology and popularization of intelligent terminals, multi-source heterogeneous data are growing rapidly. The effective utilization of rich semantic information contained in massive amounts of multi-source heterogeneous data to provide users with high-quality cross-modal information retrieval services has become an urgent problem to be solved in the current field of information retrieval. In this paper, we propose a novel cross-modal retrieval method, named MGSGH, which deeply explores the internal correlation between data of different granularities by integrating coarse-grained global semantic information and fine-grained scene graph information to model global semantic concepts and local semantic relationship graphs within a modality respectively. By enforcing cross-modal consistency constraints and intra-modal similarity preservation, we effectively integrate the visual features of image data and semantic information of text data to overcome the heterogeneity between the two types of data. Furthermore, we propose a new method for learning hash codes directly, thereby reducing the impact of quantization loss. Our comprehensive experimental evaluation demonstrated the effectiveness and superiority of the proposed model in achieving accurate and efficient cross-modal retrieval. © 2013 IEEE. Institute of Electrical and Electronics Engineers 2024 Article PeerReviewed text en cc_by_nc_nd_4 http://psasir.upm.edu.my/id/eprint/112890/1/112890.pdf Han, Zhichao and Azman, Azreen and Khalid, Fatimah and Mustaffa, Mas Rina (2024) Multi-granularity semantic information integration graph for cross-modal hash retrieval. IEEE Access, 12. pp. 44682-44694. ISSN 2169-3536 https://ieeexplore.ieee.org/document/10477417 10.1109/ACCESS.2024.3380019 |
| spellingShingle | Han, Zhichao Azman, Azreen Khalid, Fatimah Mustaffa, Mas Rina Multi-granularity semantic information integration graph for cross-modal hash retrieval |
| title | Multi-granularity semantic information integration graph for cross-modal hash retrieval |
| title_full | Multi-granularity semantic information integration graph for cross-modal hash retrieval |
| title_fullStr | Multi-granularity semantic information integration graph for cross-modal hash retrieval |
| title_full_unstemmed | Multi-granularity semantic information integration graph for cross-modal hash retrieval |
| title_short | Multi-granularity semantic information integration graph for cross-modal hash retrieval |
| title_sort | multi granularity semantic information integration graph for cross modal hash retrieval |
| url | http://psasir.upm.edu.my/id/eprint/112890/1/112890.pdf |
| work_keys_str_mv | AT hanzhichao multigranularitysemanticinformationintegrationgraphforcrossmodalhashretrieval AT azmanazreen multigranularitysemanticinformationintegrationgraphforcrossmodalhashretrieval AT khalidfatimah multigranularitysemanticinformationintegrationgraphforcrossmodalhashretrieval AT mustaffamasrina multigranularitysemanticinformationintegrationgraphforcrossmodalhashretrieval |