Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph
Predicting and understanding pedestrian intentions is crucial for autonomous vehicles and mobile robots to navigate in a crowd. However, the movement of pedestrian is random. Pedestrian trajectory modeling needs to consider not only the past movement of pedestrians, the interaction between different...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9862988/ |
| _version_ | 1818494138697383936 |
|---|---|
| author | Duan Zhao Tao Li Xiangyu Zou Yaoyi He Lichang Zhao Hui Chen Minmin Zhuo |
| author_facet | Duan Zhao Tao Li Xiangyu Zou Yaoyi He Lichang Zhao Hui Chen Minmin Zhuo |
| author_sort | Duan Zhao |
| collection | DOAJ |
| description | Predicting and understanding pedestrian intentions is crucial for autonomous vehicles and mobile robots to navigate in a crowd. However, the movement of pedestrian is random. Pedestrian trajectory modeling needs to consider not only the past movement of pedestrians, the interaction between different pedestrians, the constraints of static obstacles in the scene, but also multi-modal of the human trajectory, which brings challenges to pedestrian trajectory prediction. Most of the existing trajectory prediction methods only consider the interaction between pedestrians in the scene, ignoring the static obstacles in the scene can also have impacts on the trajectory of pedestrian. In this paper, a scalable relative interactive spatial-temporal graph generation adversarial network architecture (RISTG-GAN) is proposed to generate a reasonable multi-modal prediction trajectory by considering the interaction effects of all agents in the scene. Our method extends recent work on trajectory prediction. First, LSTM nodes are flexibly used to model the spatial-temporal graph of human-environment interactions, and the spatial-temporal graph is converted into feed-forward differentiable feature coding, and the time attention module is proposed to capture the trajectory information in time domain and learn the time dependence in long time range. Then, we capture the relative importance of the interaction of all agents in the scene on the pedestrian trajectory through the improved relative scaled dot product attention and use the generative adversarial network architecture for training to generate reasonable pedestrian future trajectory distribution. Experiments on five commonly used real public datasets show that RISTG-GAN is better than previous work in terms of reasoning speed, accuracy and the rationality of trajectory prediction. |
| first_indexed | 2024-12-10T18:02:37Z |
| format | Article |
| id | doaj.art-dab1e755ba4640308842bc0938e1ed75 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T18:02:37Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-dab1e755ba4640308842bc0938e1ed752022-12-22T01:38:44ZengIEEEIEEE Access2169-35362022-01-0110887078871810.1109/ACCESS.2022.32000669862988Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal GraphDuan Zhao0Tao Li1https://orcid.org/0000-0002-5097-2380Xiangyu Zou2https://orcid.org/0000-0002-2993-9498Yaoyi He3Lichang Zhao4Hui Chen5Minmin Zhuo6School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, ChinaSchool of Information and Control Engineering, China University of Mining and Technology, Xuzhou, ChinaSchool of Information and Control Engineering, China University of Mining and Technology, Xuzhou, ChinaTiandi (Changzhou) Automation Company Ltd., Changzhou, ChinaTiandi (Changzhou) Automation Company Ltd., Changzhou, ChinaTiandi (Changzhou) Automation Company Ltd., Changzhou, ChinaTiandi (Changzhou) Automation Company Ltd., Changzhou, ChinaPredicting and understanding pedestrian intentions is crucial for autonomous vehicles and mobile robots to navigate in a crowd. However, the movement of pedestrian is random. Pedestrian trajectory modeling needs to consider not only the past movement of pedestrians, the interaction between different pedestrians, the constraints of static obstacles in the scene, but also multi-modal of the human trajectory, which brings challenges to pedestrian trajectory prediction. Most of the existing trajectory prediction methods only consider the interaction between pedestrians in the scene, ignoring the static obstacles in the scene can also have impacts on the trajectory of pedestrian. In this paper, a scalable relative interactive spatial-temporal graph generation adversarial network architecture (RISTG-GAN) is proposed to generate a reasonable multi-modal prediction trajectory by considering the interaction effects of all agents in the scene. Our method extends recent work on trajectory prediction. First, LSTM nodes are flexibly used to model the spatial-temporal graph of human-environment interactions, and the spatial-temporal graph is converted into feed-forward differentiable feature coding, and the time attention module is proposed to capture the trajectory information in time domain and learn the time dependence in long time range. Then, we capture the relative importance of the interaction of all agents in the scene on the pedestrian trajectory through the improved relative scaled dot product attention and use the generative adversarial network architecture for training to generate reasonable pedestrian future trajectory distribution. Experiments on five commonly used real public datasets show that RISTG-GAN is better than previous work in terms of reasoning speed, accuracy and the rationality of trajectory prediction.https://ieeexplore.ieee.org/document/9862988/Pedestrian trajectory predictionspatial-temporal graphtime attentionrelative scaled dot product attentiongenerative adversarial network |
| spellingShingle | Duan Zhao Tao Li Xiangyu Zou Yaoyi He Lichang Zhao Hui Chen Minmin Zhuo Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph IEEE Access Pedestrian trajectory prediction spatial-temporal graph time attention relative scaled dot product attention generative adversarial network |
| title | Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph |
| title_full | Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph |
| title_fullStr | Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph |
| title_full_unstemmed | Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph |
| title_short | Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph |
| title_sort | multimodal pedestrian trajectory prediction based on relative interactive spatial temporal graph |
| topic | Pedestrian trajectory prediction spatial-temporal graph time attention relative scaled dot product attention generative adversarial network |
| url | https://ieeexplore.ieee.org/document/9862988/ |
| work_keys_str_mv | AT duanzhao multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph AT taoli multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph AT xiangyuzou multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph AT yaoyihe multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph AT lichangzhao multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph AT huichen multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph AT minminzhuo multimodalpedestriantrajectorypredictionbasedonrelativeinteractivespatialtemporalgraph |