Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning
© 2016 IEEE. This letter introduces a hybrid algorithm of deep reinforcement learning (RL) and Force-based motion planning (FMP) to solve distributed motion planning problem in dense and dynamic environments. Individually, RL and FMP algorithms each have their own limitations. FMP is not able to pro...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135355 |
| _version_ | 1826205621529083904 |
|---|---|
| author | Semnani, Samaneh Hosseini Liu, Hugh Everett, Michael De Ruiter, Anton How, Jonathan P |
| author2 | Massachusetts Institute of Technology. Aerospace Controls Laboratory |
| author_facet | Massachusetts Institute of Technology. Aerospace Controls Laboratory Semnani, Samaneh Hosseini Liu, Hugh Everett, Michael De Ruiter, Anton How, Jonathan P |
| author_sort | Semnani, Samaneh Hosseini |
| collection | MIT |
| description | © 2016 IEEE. This letter introduces a hybrid algorithm of deep reinforcement learning (RL) and Force-based motion planning (FMP) to solve distributed motion planning problem in dense and dynamic environments. Individually, RL and FMP algorithms each have their own limitations. FMP is not able to produce time-optimal paths and existing RL solutions are not able to produce collision-free paths in dense environments. Therefore, we first tried improving the performance of recent RL approaches by introducing a new reward function that not only eliminates the requirement of a pre supervised learning (SL) step but also decreases the chance of collision in crowded environments. That improved things, but there were still a lot of failure cases. So, we developed a hybrid approach to leverage the simpler FMP approach in stuck, simple and high-risk cases, and continue using RL for normal cases in which FMP can't produce optimal path. Also, we extend GA3C-CADRL algorithm to 3D environment. Simulation results show that the proposed algorithm outperforms both deep RL and FMP algorithms and produces up to 50$\%$ more successful scenarios than deep RL and up to 75$\%$ less extra time to reach goal than FMP. |
| first_indexed | 2024-09-23T13:16:00Z |
| format | Article |
| id | mit-1721.1/135355 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:16:00Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1353552023-03-15T20:06:15Z Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning Semnani, Samaneh Hosseini Liu, Hugh Everett, Michael De Ruiter, Anton How, Jonathan P Massachusetts Institute of Technology. Aerospace Controls Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics © 2016 IEEE. This letter introduces a hybrid algorithm of deep reinforcement learning (RL) and Force-based motion planning (FMP) to solve distributed motion planning problem in dense and dynamic environments. Individually, RL and FMP algorithms each have their own limitations. FMP is not able to produce time-optimal paths and existing RL solutions are not able to produce collision-free paths in dense environments. Therefore, we first tried improving the performance of recent RL approaches by introducing a new reward function that not only eliminates the requirement of a pre supervised learning (SL) step but also decreases the chance of collision in crowded environments. That improved things, but there were still a lot of failure cases. So, we developed a hybrid approach to leverage the simpler FMP approach in stuck, simple and high-risk cases, and continue using RL for normal cases in which FMP can't produce optimal path. Also, we extend GA3C-CADRL algorithm to 3D environment. Simulation results show that the proposed algorithm outperforms both deep RL and FMP algorithms and produces up to 50$\%$ more successful scenarios than deep RL and up to 75$\%$ less extra time to reach goal than FMP. 2021-10-27T20:23:06Z 2021-10-27T20:23:06Z 2020 2021-04-30T14:34:12Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135355 en 10.1109/LRA.2020.2974695 IEEE Robotics and Automation Letters Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Semnani, Samaneh Hosseini Liu, Hugh Everett, Michael De Ruiter, Anton How, Jonathan P Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning |
| title | Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning |
| title_full | Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning |
| title_fullStr | Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning |
| title_full_unstemmed | Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning |
| title_short | Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning |
| title_sort | multi agent motion planning for dense and dynamic environments via deep reinforcement learning |
| url | https://hdl.handle.net/1721.1/135355 |
| work_keys_str_mv | AT semnanisamanehhosseini multiagentmotionplanningfordenseanddynamicenvironmentsviadeepreinforcementlearning AT liuhugh multiagentmotionplanningfordenseanddynamicenvironmentsviadeepreinforcementlearning AT everettmichael multiagentmotionplanningfordenseanddynamicenvironmentsviadeepreinforcementlearning AT deruiteranton multiagentmotionplanningfordenseanddynamicenvironmentsviadeepreinforcementlearning AT howjonathanp multiagentmotionplanningfordenseanddynamicenvironmentsviadeepreinforcementlearning |