SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation
People’s interactions with each other form the social relations in society. Understanding human social relations in the public space is of great importance for supporting the public administrations. Recognizing social relations through visual data captured by remote sensing cameras is one of the mos...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/11/2038 |
| _version_ | 1827691827584565248 |
|---|---|
| author | Linbo Qing Lindong Li Yuchen Wang Yongqiang Cheng Yonghong Peng |
| author_facet | Linbo Qing Lindong Li Yuchen Wang Yongqiang Cheng Yonghong Peng |
| author_sort | Linbo Qing |
| collection | DOAJ |
| description | People’s interactions with each other form the social relations in society. Understanding human social relations in the public space is of great importance for supporting the public administrations. Recognizing social relations through visual data captured by remote sensing cameras is one of the most efficient ways to observe human interactions in a public space. Generally speaking, persons in the same scene tend to know each other, and the relations between person pairs are strongly correlated. The scene information in which people interact is also one of the important cues for social relation recognition (SRR). The existing works have not explored the correlations between the scene information and people’s interactions. The scene information has only been extracted on a simple level and high level semantic features to support social relation understanding are lacking. To address this issue, we propose a social relation structure-aware local–global model for SRR to exploit the high-level semantic global information of the scene where the social relation structure is explored. In our proposed model, the graph neural networks (GNNs) are employed to reason through the interactions (local information) between social relations and the global contextual information contained in the constructed scene-relation graph. Experiments demonstrate that our proposed local–global information-reasoned social relation recognition model (SRR-LGR) can reason through the local–global information. Further, the results of the final model show that our method outperforms the state-of-the-art methods. In addition, we have further discussed whether the global information contributes equally to different social relations in the same scene, by exploiting an attention mechanism in our proposed model. Further applications of SRR for human-observation are also exploited. |
| first_indexed | 2024-03-10T11:10:03Z |
| format | Article |
| id | doaj.art-2aaf199a1479492fb9564591f9640a6c |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T11:10:03Z |
| publishDate | 2021-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-2aaf199a1479492fb9564591f9640a6c2023-11-21T20:50:01ZengMDPI AGRemote Sensing2072-42922021-05-011311203810.3390/rs13112038SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented ObservationLinbo Qing0Lindong Li1Yuchen Wang2Yongqiang Cheng3Yonghong Peng4College Electronics and Information Engineering, Sichuan Universicty, Chengdu 610065, ChinaCollege Electronics and Information Engineering, Sichuan Universicty, Chengdu 610065, ChinaCollege Electronics and Information Engineering, Sichuan Universicty, Chengdu 610065, ChinaDepartment of Computer Science and Technology, University of Hull, Hull HU67RX, UKDepartment of Computing and Mathematics, Manchester Metropolitan University, Manchester M156BH, UKPeople’s interactions with each other form the social relations in society. Understanding human social relations in the public space is of great importance for supporting the public administrations. Recognizing social relations through visual data captured by remote sensing cameras is one of the most efficient ways to observe human interactions in a public space. Generally speaking, persons in the same scene tend to know each other, and the relations between person pairs are strongly correlated. The scene information in which people interact is also one of the important cues for social relation recognition (SRR). The existing works have not explored the correlations between the scene information and people’s interactions. The scene information has only been extracted on a simple level and high level semantic features to support social relation understanding are lacking. To address this issue, we propose a social relation structure-aware local–global model for SRR to exploit the high-level semantic global information of the scene where the social relation structure is explored. In our proposed model, the graph neural networks (GNNs) are employed to reason through the interactions (local information) between social relations and the global contextual information contained in the constructed scene-relation graph. Experiments demonstrate that our proposed local–global information-reasoned social relation recognition model (SRR-LGR) can reason through the local–global information. Further, the results of the final model show that our method outperforms the state-of-the-art methods. In addition, we have further discussed whether the global information contributes equally to different social relations in the same scene, by exploiting an attention mechanism in our proposed model. Further applications of SRR for human-observation are also exploited.https://www.mdpi.com/2072-4292/13/11/2038social relation recognitionremote sensing camerahuman-oriented observationlocal–global informationgraph neural networks |
| spellingShingle | Linbo Qing Lindong Li Yuchen Wang Yongqiang Cheng Yonghong Peng SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation Remote Sensing social relation recognition remote sensing camera human-oriented observation local–global information graph neural networks |
| title | SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation |
| title_full | SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation |
| title_fullStr | SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation |
| title_full_unstemmed | SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation |
| title_short | SRR-LGR: Local–Global Information-Reasoned Social Relation Recognition for Human-Oriented Observation |
| title_sort | srr lgr local global information reasoned social relation recognition for human oriented observation |
| topic | social relation recognition remote sensing camera human-oriented observation local–global information graph neural networks |
| url | https://www.mdpi.com/2072-4292/13/11/2038 |
| work_keys_str_mv | AT linboqing srrlgrlocalglobalinformationreasonedsocialrelationrecognitionforhumanorientedobservation AT lindongli srrlgrlocalglobalinformationreasonedsocialrelationrecognitionforhumanorientedobservation AT yuchenwang srrlgrlocalglobalinformationreasonedsocialrelationrecognitionforhumanorientedobservation AT yongqiangcheng srrlgrlocalglobalinformationreasonedsocialrelationrecognitionforhumanorientedobservation AT yonghongpeng srrlgrlocalglobalinformationreasonedsocialrelationrecognitionforhumanorientedobservation |