Trajectory Prediction with Linguistic Representations
2022 IEEE International Conference on Robotics and Automation (ICRA) May 23-27, 2022. Philadelphia, PA, USA
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024
|
| Online Access: | https://hdl.handle.net/1721.1/153753 |
| _version_ | 1824458279921647616 |
|---|---|
| author | Kuo, Yen-Ling Huang, Xin Barbu, Andrei McGill, Stephen G. Katz, Boris Leonard, John J. Rosman, Guy |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Kuo, Yen-Ling Huang, Xin Barbu, Andrei McGill, Stephen G. Katz, Boris Leonard, John J. Rosman, Guy |
| author_sort | Kuo, Yen-Ling |
| collection | MIT |
| description | 2022 IEEE International Conference on Robotics and Automation (ICRA) May 23-27, 2022. Philadelphia, PA, USA |
| first_indexed | 2024-09-23T14:05:45Z |
| format | Article |
| id | mit-1721.1/153753 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2025-02-19T04:23:22Z |
| publishDate | 2024 |
| publisher | IEEE |
| record_format | dspace |
| spelling | mit-1721.1/1537532024-11-05T15:15:22Z Trajectory Prediction with Linguistic Representations Kuo, Yen-Ling Huang, Xin Barbu, Andrei McGill, Stephen G. Katz, Boris Leonard, John J. Rosman, Guy Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Center for Brains, Minds, and Machines 2022 IEEE International Conference on Robotics and Automation (ICRA) May 23-27, 2022. Philadelphia, PA, USA Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions. We present a novel trajectory prediction model that uses linguistic intermediate representations to forecast trajectories, and is trained using trajectory samples with partially-annotated captions. The model learns the meaning of each of the words without direct per-word supervision. At inference time, it generates a linguistic description of trajectories which captures maneuvers and interactions over an extended time interval. This generated description is used to refine predictions of the trajectories of multiple agents. We train and validate our model on the Argoverse dataset, and demonstrate improved accuracy results in trajectory prediction. In addition, our model is more interpretable: it presents part of its reasoning in plain language as captions, which can aid model development and can aid in building confidence in the model before deploying it. 2024-03-14T17:10:45Z 2024-03-14T17:10:45Z 2022-05-23 2024-03-14T16:53:35Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/153753 Kuo, Yen-Ling, Huang, Xin, Barbu, Andrei, McGill, Stephen G., Katz, Boris et al. 2022. "Trajectory Prediction with Linguistic Representations." en 10.1109/icra46639.2022.9811928 Creative Commons Attribution-Noncommercial-ShareAlike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf IEEE arxiv |
| spellingShingle | Kuo, Yen-Ling Huang, Xin Barbu, Andrei McGill, Stephen G. Katz, Boris Leonard, John J. Rosman, Guy Trajectory Prediction with Linguistic Representations |
| title | Trajectory Prediction with Linguistic Representations |
| title_full | Trajectory Prediction with Linguistic Representations |
| title_fullStr | Trajectory Prediction with Linguistic Representations |
| title_full_unstemmed | Trajectory Prediction with Linguistic Representations |
| title_short | Trajectory Prediction with Linguistic Representations |
| title_sort | trajectory prediction with linguistic representations |
| url | https://hdl.handle.net/1721.1/153753 |
| work_keys_str_mv | AT kuoyenling trajectorypredictionwithlinguisticrepresentations AT huangxin trajectorypredictionwithlinguisticrepresentations AT barbuandrei trajectorypredictionwithlinguisticrepresentations AT mcgillstepheng trajectorypredictionwithlinguisticrepresentations AT katzboris trajectorypredictionwithlinguisticrepresentations AT leonardjohnj trajectorypredictionwithlinguisticrepresentations AT rosmanguy trajectorypredictionwithlinguisticrepresentations |