Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
This paper deals with the problem of recognizing the unique role in dynamic environments. Different from social roles, the unique role refers to those who are unusual in their carrying items or movements in the scene. In this paper, we propose a hierarchical probabilistic reasoning method that relat...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/106302 http://hdl.handle.net/10220/48869 |
| _version_ | 1826128421945606144 |
|---|---|
| author | Yang, Chule Yue, Yufeng Zhang, Jun Wen, Mingxing Wang, Danwei |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Yang, Chule Yue, Yufeng Zhang, Jun Wen, Mingxing Wang, Danwei |
| author_sort | Yang, Chule |
| collection | NTU |
| description | This paper deals with the problem of recognizing the unique role in dynamic environments. Different from social roles, the unique role refers to those who are unusual in their carrying items or movements in the scene. In this paper, we propose a hierarchical probabilistic reasoning method that relates spatial relationships between interested objects and humans with their temporal changes to recognize the unique individual. Two observation models, Object Existence Model (OEM) and Human Action Model (HAM), are established to support role inference by analyzing the corresponding semantic-interaction features and spatio-temporal features. Then, OEM and HAM results of each person are compared with the overall distribution in the scene, respectively. Finally, we can determine the role through the fusion of two observation models. Experiments are conducted in both indoor and outdoor environments concerning different settings, degrees of clutter, and occlusions. The results show that the proposed method can adapt to a variety of scenarios and outperforms other methods on accuracy and robustness, moreover, exhibiting stable performance even in complex scenes. |
| first_indexed | 2024-10-01T07:24:42Z |
| format | Journal Article |
| id | ntu-10356/106302 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:24:42Z |
| publishDate | 2019 |
| record_format | dspace |
| spelling | ntu-10356/1063022020-06-08T09:33:34Z Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features Yang, Chule Yue, Yufeng Zhang, Jun Wen, Mingxing Wang, Danwei School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics Probabilistic Logic Semantics This paper deals with the problem of recognizing the unique role in dynamic environments. Different from social roles, the unique role refers to those who are unusual in their carrying items or movements in the scene. In this paper, we propose a hierarchical probabilistic reasoning method that relates spatial relationships between interested objects and humans with their temporal changes to recognize the unique individual. Two observation models, Object Existence Model (OEM) and Human Action Model (HAM), are established to support role inference by analyzing the corresponding semantic-interaction features and spatio-temporal features. Then, OEM and HAM results of each person are compared with the overall distribution in the scene, respectively. Finally, we can determine the role through the fusion of two observation models. Experiments are conducted in both indoor and outdoor environments concerning different settings, degrees of clutter, and occlusions. The results show that the proposed method can adapt to a variety of scenarios and outperforms other methods on accuracy and robustness, moreover, exhibiting stable performance even in complex scenes. NRF (Natl Research Foundation, S’pore) Accepted version 2019-06-20T05:16:48Z 2019-12-06T22:08:34Z 2019-06-20T05:16:48Z 2019-12-06T22:08:34Z 2019 2019 Journal Article Yang, C., Yue, Y., Zhang, J., Wen, M., & Wang, D. (2019). Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features. IEEE Transactions on Multimedia, 21(5), 1195-1208. doi:10.1109/TMM.2018.2875513 1520-9210 https://hdl.handle.net/10356/106302 http://hdl.handle.net/10220/48869 10.1109/TMM.2018.2875513 210580 5 21 1195 1208 210580 en IEEE Transactions on Multimedia © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/TMM.2018.2875513 14 p. application/pdf |
| spellingShingle | Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics Probabilistic Logic Semantics Yang, Chule Yue, Yufeng Zhang, Jun Wen, Mingxing Wang, Danwei Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features |
| title | Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features |
| title_full | Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features |
| title_fullStr | Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features |
| title_full_unstemmed | Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features |
| title_short | Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features |
| title_sort | probabilistic reasoning for unique role recognition based on the fusion of semantic interaction and spatio temporal features |
| topic | Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics Probabilistic Logic Semantics |
| url | https://hdl.handle.net/10356/106302 http://hdl.handle.net/10220/48869 |
| work_keys_str_mv | AT yangchule probabilisticreasoningforuniquerolerecognitionbasedonthefusionofsemanticinteractionandspatiotemporalfeatures AT yueyufeng probabilisticreasoningforuniquerolerecognitionbasedonthefusionofsemanticinteractionandspatiotemporalfeatures AT zhangjun probabilisticreasoningforuniquerolerecognitionbasedonthefusionofsemanticinteractionandspatiotemporalfeatures AT wenmingxing probabilisticreasoningforuniquerolerecognitionbasedonthefusionofsemanticinteractionandspatiotemporalfeatures AT wangdanwei probabilisticreasoningforuniquerolerecognitionbasedonthefusionofsemanticinteractionandspatiotemporalfeatures |