Image-based social relation recognition using graph neural network
Social relation, which indicates how people are connected in society, is an essential part of our social life. With the boom of social media, data like pictures, videos, and texts, become available and can be used for social relation recognition (SRR). Meantime, the advancement of computing infrastr...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Coursework |
| Language: | English |
| Published: |
Nanyang Technological University
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/149390 |
| _version_ | 1811697737704407040 |
|---|---|
| author | Gao, Jianjun |
| author2 | Yap Kim Hui |
| author_facet | Yap Kim Hui Gao, Jianjun |
| author_sort | Gao, Jianjun |
| collection | NTU |
| description | Social relation, which indicates how people are connected in society, is an essential part of our social life. With the boom of social media, data like pictures, videos, and texts, become available and can be used for social relation recognition (SRR). Meantime, the advancement of computing infrastructure and computer vision research in recent years has made it possible for computers to process these kinds of data to recognize social relations in our life. SRR is a complex topic as social relations among humans diverse a lot. Because of Convolutional Neural Network (CNN) and Graph Neural Network (GNN), it has become possible for a machine to recognize social relations in an acceptable condition. SRR problems were mostly solved by feature extraction and graph reasoning process which depend on CNN and GNN respectively. In this work, the proposed SRR was based on the Interpersonal Relation benchmark dataset [1]. Also, following existing work, we extracted features from multi-scale views from images and reasoned by two-directional graphs with Gated Recurrent Unit (GRU) attention mechanism. The results showed that the proposed work surpasses the state-of-the-art work on the Interpersonal Relation dataset by about 10% in balanced accuracy. |
| first_indexed | 2024-10-01T08:00:01Z |
| format | Thesis-Master by Coursework |
| id | ntu-10356/149390 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T08:00:01Z |
| publishDate | 2021 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1493902023-07-04T17:02:08Z Image-based social relation recognition using graph neural network Gao, Jianjun Yap Kim Hui School of Electrical and Electronic Engineering EKHYap@ntu.edu.sg Engineering::Electrical and electronic engineering Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Social relation, which indicates how people are connected in society, is an essential part of our social life. With the boom of social media, data like pictures, videos, and texts, become available and can be used for social relation recognition (SRR). Meantime, the advancement of computing infrastructure and computer vision research in recent years has made it possible for computers to process these kinds of data to recognize social relations in our life. SRR is a complex topic as social relations among humans diverse a lot. Because of Convolutional Neural Network (CNN) and Graph Neural Network (GNN), it has become possible for a machine to recognize social relations in an acceptable condition. SRR problems were mostly solved by feature extraction and graph reasoning process which depend on CNN and GNN respectively. In this work, the proposed SRR was based on the Interpersonal Relation benchmark dataset [1]. Also, following existing work, we extracted features from multi-scale views from images and reasoned by two-directional graphs with Gated Recurrent Unit (GRU) attention mechanism. The results showed that the proposed work surpasses the state-of-the-art work on the Interpersonal Relation dataset by about 10% in balanced accuracy. Master of Science (Communications Engineering) 2021-05-19T04:32:28Z 2021-05-19T04:32:28Z 2021 Thesis-Master by Coursework Gao, J. (2021). Image-based social relation recognition using graph neural network. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/149390 https://hdl.handle.net/10356/149390 en application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Gao, Jianjun Image-based social relation recognition using graph neural network |
| title | Image-based social relation recognition using graph neural network |
| title_full | Image-based social relation recognition using graph neural network |
| title_fullStr | Image-based social relation recognition using graph neural network |
| title_full_unstemmed | Image-based social relation recognition using graph neural network |
| title_short | Image-based social relation recognition using graph neural network |
| title_sort | image based social relation recognition using graph neural network |
| topic | Engineering::Electrical and electronic engineering Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision |
| url | https://hdl.handle.net/10356/149390 |
| work_keys_str_mv | AT gaojianjun imagebasedsocialrelationrecognitionusinggraphneuralnetwork |