Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning
© 2013 IEEE. Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/133362 |
| _version_ | 1826204380136734720 |
|---|---|
| author | Everett, Michael Chen, Yu Fan How, Jonathan P |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Everett, Michael Chen, Yu Fan How, Jonathan P |
| author_sort | Everett, Michael |
| collection | MIT |
| description | © 2013 IEEE. Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and other robots. Existing RL-based works assume homogeneity of agent properties, use specific motion models over short timescales, or lack a principled method to handle a large, possibly varying number of agents. Therefore, this work develops an algorithm that learns collision avoidance among a variety of heterogeneous, non-communicating, dynamic agents without assuming they follow any particular behavior rules. It extends our previous work by introducing a strategy using Long Short-Term Memory (LSTM) that enables the algorithm to use observations of an arbitrary number of other agents, instead of a small, fixed number of neighbors. The proposed algorithm is shown to outperform a classical collision avoidance algorithm, another deep RL-based algorithm, and scales with the number of agents better (fewer collisions, shorter time to goal) than our previously published learning-based approach. Analysis of the LSTM provides insights into how observations of nearby agents affect the hidden state and quantifies the performance impact of various agent ordering heuristics. The learned policy generalizes to several applications beyond the training scenarios: formation control (arrangement into letters), demonstrations on a fleet of four multirotors and on a fully autonomous robotic vehicle capable of traveling at human walking speed among pedestrians. |
| first_indexed | 2024-09-23T12:53:48Z |
| format | Article |
| id | mit-1721.1/133362 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T12:53:48Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1333622023-09-12T19:53:30Z Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning Everett, Michael Chen, Yu Fan How, Jonathan P Massachusetts Institute of Technology. Department of Aeronautics and Astronautics © 2013 IEEE. Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and other robots. Existing RL-based works assume homogeneity of agent properties, use specific motion models over short timescales, or lack a principled method to handle a large, possibly varying number of agents. Therefore, this work develops an algorithm that learns collision avoidance among a variety of heterogeneous, non-communicating, dynamic agents without assuming they follow any particular behavior rules. It extends our previous work by introducing a strategy using Long Short-Term Memory (LSTM) that enables the algorithm to use observations of an arbitrary number of other agents, instead of a small, fixed number of neighbors. The proposed algorithm is shown to outperform a classical collision avoidance algorithm, another deep RL-based algorithm, and scales with the number of agents better (fewer collisions, shorter time to goal) than our previously published learning-based approach. Analysis of the LSTM provides insights into how observations of nearby agents affect the hidden state and quantifies the performance impact of various agent ordering heuristics. The learned policy generalizes to several applications beyond the training scenarios: formation control (arrangement into letters), demonstrations on a fleet of four multirotors and on a fully autonomous robotic vehicle capable of traveling at human walking speed among pedestrians. 2021-10-27T19:52:21Z 2021-10-27T19:52:21Z 2021 2021-04-30T16:33:14Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/133362 en 10.1109/ACCESS.2021.3050338 IEEE Access Creative Commons Attribution 4.0 International license https://creativecommons.org/licenses/by/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) IEEE |
| spellingShingle | Everett, Michael Chen, Yu Fan How, Jonathan P Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning |
| title | Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning |
| title_full | Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning |
| title_fullStr | Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning |
| title_full_unstemmed | Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning |
| title_short | Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning |
| title_sort | collision avoidance in pedestrian rich environments with deep reinforcement learning |
| url | https://hdl.handle.net/1721.1/133362 |
| work_keys_str_mv | AT everettmichael collisionavoidanceinpedestrianrichenvironmentswithdeepreinforcementlearning AT chenyufan collisionavoidanceinpedestrianrichenvironmentswithdeepreinforcementlearning AT howjonathanp collisionavoidanceinpedestrianrichenvironmentswithdeepreinforcementlearning |