Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection
Human–object interaction (HOI) detection is important for promoting the development of many fields such as human–computer interactions, service robotics, and video security surveillance. A high percentage of human–object pairs with invalid interactions are discovered in the object detection phase of...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI
2023
|
| Subjects: | |
| Online Access: | http://eprints.utm.my/106602/1/ZuriahatiMohdYunos2023_InteractivityRecognitionGraphNeuralNetwork.pdf |
| _version_ | 1811132371292913664 |
|---|---|
| author | Zhang, Jiali Mohd. Yunos, Zuriahati Haron, Habibollah |
| author_facet | Zhang, Jiali Mohd. Yunos, Zuriahati Haron, Habibollah |
| author_sort | Zhang, Jiali |
| collection | ePrints |
| description | Human–object interaction (HOI) detection is important for promoting the development of many fields such as human–computer interactions, service robotics, and video security surveillance. A high percentage of human–object pairs with invalid interactions are discovered in the object detection phase of conventional human–object interaction detection algorithms, resulting in inaccurate interaction detection. To recognize invalid human–object interaction pairs, this paper proposes a model structure, the interactivity recognition graph neural network (IR-GNN) model, which can directly infer the probability of human–object interactions from a graph model architecture. The model consists of three modules: The first one is the human posture feature module, which uses key points of the human body to construct relative spatial pose features and further facilitates the discrimination of human–object interactivity through human pose information. Second, a human–object interactivity graph module is proposed. The spatial relationship of human–object distance is used as the initialization weight of edges, and the graph is updated by combining the message passing of attention mechanism so that edges with interacting node pairs obtain higher weights. Thirdly, the classification module is proposed, by finally using a fully connected neural network, the interactivity of human–object pairs is binarily classified. These three modules work in collaboration to enable the effective inference of interactive possibilities. On the datasets HICO-DET and V-COCO, comparative and ablation experiments are carried out. It has been proved that our technology can improve the detection of human–object interactions. |
| first_indexed | 2024-09-24T00:03:01Z |
| format | Article |
| id | utm.eprints-106602 |
| institution | Universiti Teknologi Malaysia - ePrints |
| language | English |
| last_indexed | 2024-09-24T00:03:01Z |
| publishDate | 2023 |
| publisher | MDPI |
| record_format | dspace |
| spelling | utm.eprints-1066022024-07-14T09:19:05Z http://eprints.utm.my/106602/ Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection Zhang, Jiali Mohd. Yunos, Zuriahati Haron, Habibollah QA75 Electronic computers. Computer science Human–object interaction (HOI) detection is important for promoting the development of many fields such as human–computer interactions, service robotics, and video security surveillance. A high percentage of human–object pairs with invalid interactions are discovered in the object detection phase of conventional human–object interaction detection algorithms, resulting in inaccurate interaction detection. To recognize invalid human–object interaction pairs, this paper proposes a model structure, the interactivity recognition graph neural network (IR-GNN) model, which can directly infer the probability of human–object interactions from a graph model architecture. The model consists of three modules: The first one is the human posture feature module, which uses key points of the human body to construct relative spatial pose features and further facilitates the discrimination of human–object interactivity through human pose information. Second, a human–object interactivity graph module is proposed. The spatial relationship of human–object distance is used as the initialization weight of edges, and the graph is updated by combining the message passing of attention mechanism so that edges with interacting node pairs obtain higher weights. Thirdly, the classification module is proposed, by finally using a fully connected neural network, the interactivity of human–object pairs is binarily classified. These three modules work in collaboration to enable the effective inference of interactive possibilities. On the datasets HICO-DET and V-COCO, comparative and ablation experiments are carried out. It has been proved that our technology can improve the detection of human–object interactions. MDPI 2023 Article PeerReviewed application/pdf en http://eprints.utm.my/106602/1/ZuriahatiMohdYunos2023_InteractivityRecognitionGraphNeuralNetwork.pdf Zhang, Jiali and Mohd. Yunos, Zuriahati and Haron, Habibollah (2023) Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection. Electronics (Switzerland), 12 (2). pp. 1-19. ISSN 2079-9292 http://dx.doi.org/10.3390/electronics12020470 DOI : 10.3390/electronics12020470 |
| spellingShingle | QA75 Electronic computers. Computer science Zhang, Jiali Mohd. Yunos, Zuriahati Haron, Habibollah Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection |
| title | Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection |
| title_full | Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection |
| title_fullStr | Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection |
| title_full_unstemmed | Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection |
| title_short | Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection |
| title_sort | interactivity recognition graph neural network ir gnn model for improving human object interaction detection |
| topic | QA75 Electronic computers. Computer science |
| url | http://eprints.utm.my/106602/1/ZuriahatiMohdYunos2023_InteractivityRecognitionGraphNeuralNetwork.pdf |
| work_keys_str_mv | AT zhangjiali interactivityrecognitiongraphneuralnetworkirgnnmodelforimprovinghumanobjectinteractiondetection AT mohdyunoszuriahati interactivityrecognitiongraphneuralnetworkirgnnmodelforimprovinghumanobjectinteractiondetection AT haronhabibollah interactivityrecognitiongraphneuralnetworkirgnnmodelforimprovinghumanobjectinteractiondetection |