Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions
The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems, but representing and solving Dec-POMDPs is often in...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2018
|
| Online Access: | http://hdl.handle.net/1721.1/116391 https://orcid.org/0000-0003-0903-0137 https://orcid.org/0000-0002-6786-7384 https://orcid.org/0000-0001-8576-1930 |
| _version_ | 1826216774900645888 |
|---|---|
| author | Omidshafiei, Shayegan Aghamohammadi, Aliakbar Amato, Christopher How, Jonathan P |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Omidshafiei, Shayegan Aghamohammadi, Aliakbar Amato, Christopher How, Jonathan P |
| author_sort | Omidshafiei, Shayegan |
| collection | MIT |
| description | The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems, but representing and solving Dec-POMDPs is often intractable for large problems. To allow for a high-level representation that is natural for multi-robot problems and scalable to large discrete and continuous problems, this paper extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP). The Dec-POSMDP formulation allows asynchronous decision-making by the robots, which is crucial in multi-robot domains. We also present an algorithm for solving this Dec-POSMDP which is much more scalable than previous methods since it can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed method's performance is evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent multi-robot problems and provide high-quality solutions for large-scale problems. |
| first_indexed | 2024-09-23T16:52:54Z |
| format | Article |
| id | mit-1721.1/116391 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:52:54Z |
| publishDate | 2018 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1163912022-09-29T22:10:03Z Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions Omidshafiei, Shayegan Aghamohammadi, Aliakbar Amato, Christopher How, Jonathan P Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Omidshafiei, Shayegan Aghamohammadi, Aliakbar Amato, Christopher How, Jonathan P The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems, but representing and solving Dec-POMDPs is often intractable for large problems. To allow for a high-level representation that is natural for multi-robot problems and scalable to large discrete and continuous problems, this paper extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP). The Dec-POSMDP formulation allows asynchronous decision-making by the robots, which is crucial in multi-robot domains. We also present an algorithm for solving this Dec-POSMDP which is much more scalable than previous methods since it can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed method's performance is evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent multi-robot problems and provide high-quality solutions for large-scale problems. 2018-06-19T12:32:50Z 2018-06-19T12:32:50Z 2015-07 2018-03-22T12:31:07Z Article http://purl.org/eprint/type/ConferencePaper 978-1-4799-6923-4 http://hdl.handle.net/1721.1/116391 Omidshafiei, Shayegan, Ali-akbar Agha-mohammadi, Christopher Amato, and Jonathan P. How. “Decentralized Control of Partially Observable Markov Decision Processes Using Belief Space Macro-Actions.” 2015 IEEE International Conference on Robotics and Automation (ICRA) (May 2015). https://orcid.org/0000-0003-0903-0137 https://orcid.org/0000-0002-6786-7384 https://orcid.org/0000-0001-8576-1930 http://dx.doi.org/10.1109/ICRA.2015.7140035 2015 IEEE International Conference on Robotics and Automation (ICRA) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) Other repository |
| spellingShingle | Omidshafiei, Shayegan Aghamohammadi, Aliakbar Amato, Christopher How, Jonathan P Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions |
| title | Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions |
| title_full | Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions |
| title_fullStr | Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions |
| title_full_unstemmed | Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions |
| title_short | Decentralized control of Partially Observable Markov Decision Processes using belief space macro-actions |
| title_sort | decentralized control of partially observable markov decision processes using belief space macro actions |
| url | http://hdl.handle.net/1721.1/116391 https://orcid.org/0000-0003-0903-0137 https://orcid.org/0000-0002-6786-7384 https://orcid.org/0000-0001-8576-1930 |
| work_keys_str_mv | AT omidshafieishayegan decentralizedcontrolofpartiallyobservablemarkovdecisionprocessesusingbeliefspacemacroactions AT aghamohammadialiakbar decentralizedcontrolofpartiallyobservablemarkovdecisionprocessesusingbeliefspacemacroactions AT amatochristopher decentralizedcontrolofpartiallyobservablemarkovdecisionprocessesusingbeliefspacemacroactions AT howjonathanp decentralizedcontrolofpartiallyobservablemarkovdecisionprocessesusingbeliefspacemacroactions |