Visual relationship detection with contextual information
Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined b...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/146883 |
| _version_ | 1826112237235863552 |
|---|---|
| author | Li, Yugang Wang, Yongbin Chen, Zhe Zhu, Yuting |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Li, Yugang Wang, Yongbin Chen, Zhe Zhu, Yuting |
| author_sort | Li, Yugang |
| collection | NTU |
| description | Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined by the surrounding objects and other contextual information. In this work, we employ this insight to propose a novel framework to deal with the problem of visual relationship detection. The core of the framework is a relationship inference network, which is a recurrent structure designed for combining the global contextual information of the object to infer the relationship of the image. Experimental results on Stanford VRD and Visual Genome demonstrate that the proposed method achieves a good performance both in efficiency and accuracy. Finally, we demonstrate the value of visual relationship on two computer vision tasks: image retrieval and scene graph generation. |
| first_indexed | 2024-10-01T03:04:00Z |
| format | Journal Article |
| id | ntu-10356/146883 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:04:00Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | ntu-10356/1468832021-03-12T06:34:55Z Visual relationship detection with contextual information Li, Yugang Wang, Yongbin Chen, Zhe Zhu, Yuting School of Electrical and Electronic Engineering Engineering::Computer science and engineering Visual Relationship Deep Learning Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined by the surrounding objects and other contextual information. In this work, we employ this insight to propose a novel framework to deal with the problem of visual relationship detection. The core of the framework is a relationship inference network, which is a recurrent structure designed for combining the global contextual information of the object to infer the relationship of the image. Experimental results on Stanford VRD and Visual Genome demonstrate that the proposed method achieves a good performance both in efficiency and accuracy. Finally, we demonstrate the value of visual relationship on two computer vision tasks: image retrieval and scene graph generation. Published version 2021-03-12T06:34:54Z 2021-03-12T06:34:54Z 2020 Journal Article Li, Y., Wang, Y., Chen, Z. & Zhu, Y. (2020). Visual relationship detection with contextual information. Computers, Materials and Continua, 63(3), 1575-1589. https://dx.doi.org/10.32604/CMC.2020.07451 1546-2218 https://hdl.handle.net/10356/146883 10.32604/CMC.2020.07451 2-s2.0-85091087063 3 63 1575 1589 en Computers, Materials and Continua © 2020 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. application/pdf |
| spellingShingle | Engineering::Computer science and engineering Visual Relationship Deep Learning Li, Yugang Wang, Yongbin Chen, Zhe Zhu, Yuting Visual relationship detection with contextual information |
| title | Visual relationship detection with contextual information |
| title_full | Visual relationship detection with contextual information |
| title_fullStr | Visual relationship detection with contextual information |
| title_full_unstemmed | Visual relationship detection with contextual information |
| title_short | Visual relationship detection with contextual information |
| title_sort | visual relationship detection with contextual information |
| topic | Engineering::Computer science and engineering Visual Relationship Deep Learning |
| url | https://hdl.handle.net/10356/146883 |
| work_keys_str_mv | AT liyugang visualrelationshipdetectionwithcontextualinformation AT wangyongbin visualrelationshipdetectionwithcontextualinformation AT chenzhe visualrelationshipdetectionwithcontextualinformation AT zhuyuting visualrelationshipdetectionwithcontextualinformation |