Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks
Predicting the multimodal future motions of neighboring agents is essential for an autonomous vehicle to navigate complex scenarios. It is challenging as the motion of an agent is affected by the complex interaction among itself, other agents, and the local roads. Unlike most existing works, which p...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/170248 |
| _version_ | 1826110447934242816 |
|---|---|
| author | Mo, Xiaoyu Xing, Yang Liu, Haochen Lv, Chen |
| author2 | School of Mechanical and Aerospace Engineering |
| author_facet | School of Mechanical and Aerospace Engineering Mo, Xiaoyu Xing, Yang Liu, Haochen Lv, Chen |
| author_sort | Mo, Xiaoyu |
| collection | NTU |
| description | Predicting the multimodal future motions of neighboring agents is essential for an autonomous vehicle to navigate complex scenarios. It is challenging as the motion of an agent is affected by the complex interaction among itself, other agents, and the local roads. Unlike most existing works, which predict a fixed number of possible future motions of an agent, we propose a map-adaptive predictor that can predict a variable number of future trajectories of an agent according to the number of lanes with candidate centerlines (CCLs). The predictor predicts not only future motions guided by single CCLs but also a scene-reasoning prediction and a motion-maintaining prediction. These three kinds of predictions are produced integrally via a single graph operation. We represent the driving scene with a heterogeneous hierarchical graph containing nodes of two types. An agent node contains its dynamics feature encoded from its historical states, and a CCL node contains the CCL's sequential feature. We propose a hierarchical graph operator (HGO) with an edge-masking technology to regulate the information flow in graph operations and obtain the encoded scene feature for the trajectory decoder. Experiments on two large-scale real-world driving datasets show that our method realizes map-adaptive prediction and outperforms strong baselines. |
| first_indexed | 2024-10-01T02:34:35Z |
| format | Journal Article |
| id | ntu-10356/170248 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:34:35Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1702482023-09-05T00:52:25Z Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks Mo, Xiaoyu Xing, Yang Liu, Haochen Lv, Chen School of Mechanical and Aerospace Engineering Engineering::Mechanical engineering Connected Vehicles Graph Neural Networks Predicting the multimodal future motions of neighboring agents is essential for an autonomous vehicle to navigate complex scenarios. It is challenging as the motion of an agent is affected by the complex interaction among itself, other agents, and the local roads. Unlike most existing works, which predict a fixed number of possible future motions of an agent, we propose a map-adaptive predictor that can predict a variable number of future trajectories of an agent according to the number of lanes with candidate centerlines (CCLs). The predictor predicts not only future motions guided by single CCLs but also a scene-reasoning prediction and a motion-maintaining prediction. These three kinds of predictions are produced integrally via a single graph operation. We represent the driving scene with a heterogeneous hierarchical graph containing nodes of two types. An agent node contains its dynamics feature encoded from its historical states, and a CCL node contains the CCL's sequential feature. We propose a hierarchical graph operator (HGO) with an edge-masking technology to regulate the information flow in graph operations and obtain the encoded scene feature for the trajectory decoder. Experiments on two large-scale real-world driving datasets show that our method realizes map-adaptive prediction and outperforms strong baselines. Nanyang Technological University This work was supported in part by SUG-NAP under Grant M4082268.050 and in part by Nanyang Technological University, Singapore. 2023-09-05T00:52:25Z 2023-09-05T00:52:25Z 2023 Journal Article Mo, X., Xing, Y., Liu, H. & Lv, C. (2023). Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks. IEEE Robotics and Automation Letters, 8(6), 3685-3692. https://dx.doi.org/10.1109/LRA.2023.3270739 2377-3766 https://hdl.handle.net/10356/170248 10.1109/LRA.2023.3270739 2-s2.0-85159672325 6 8 3685 3692 en M4082268.050 IEEE Robotics and Automation Letters © 2023 IEEE. All rights reserved. |
| spellingShingle | Engineering::Mechanical engineering Connected Vehicles Graph Neural Networks Mo, Xiaoyu Xing, Yang Liu, Haochen Lv, Chen Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| title | Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| title_full | Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| title_fullStr | Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| title_full_unstemmed | Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| title_short | Map-adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| title_sort | map adaptive multimodal trajectory prediction using hierarchical graph neural networks |
| topic | Engineering::Mechanical engineering Connected Vehicles Graph Neural Networks |
| url | https://hdl.handle.net/10356/170248 |
| work_keys_str_mv | AT moxiaoyu mapadaptivemultimodaltrajectorypredictionusinghierarchicalgraphneuralnetworks AT xingyang mapadaptivemultimodaltrajectorypredictionusinghierarchicalgraphneuralnetworks AT liuhaochen mapadaptivemultimodaltrajectorypredictionusinghierarchicalgraphneuralnetworks AT lvchen mapadaptivemultimodaltrajectorypredictionusinghierarchicalgraphneuralnetworks |