Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution
Online, forward-search techniques have demonstrated promising results for solving problems in partially observable environments. These techniques depend on the ability to efficiently search and evaluate the set of beliefs reachable from the current belief. However, enumerating or sampling action-obs...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Published: |
2009
|
| Online Access: | http://hdl.handle.net/1721.1/46820 |
| _version_ | 1826214993932058624 |
|---|---|
| author | Roy, Nicholas He, Ruijie |
| author2 | Nicholas Roy |
| author_facet | Nicholas Roy Roy, Nicholas He, Ruijie |
| author_sort | Roy, Nicholas |
| collection | MIT |
| description | Online, forward-search techniques have demonstrated promising results for solving problems in partially observable environments. These techniques depend on the ability to efficiently search and evaluate the set of beliefs reachable from the current belief. However, enumerating or sampling action-observation sequences to compute the reachable beliefs is computationally demanding; coupled with the need to satisfy real-time constraints, existing online solvers can only search to a limited depth. In this paper, we propose that policies can be generated directly from the distribution of the agent's posterior belief. When the underlying state distribution is Gaussian, and the observation function is an exponential family distribution, we can calculate this distribution of beliefs without enumerating the possible observations. This property not only enables us to plan in problems with large observation spaces, but also allows us to search deeper by considering policies composed of multi-step action sequences. We present the Posterior Belief Distribution (PBD) algorithm, an efficient forward-search POMDP planner for continuous domains, demonstrating that better policies are generated when we can perform deeper forward search. |
| first_indexed | 2024-09-23T16:15:07Z |
| id | mit-1721.1/46820 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:15:07Z |
| publishDate | 2009 |
| record_format | dspace |
| spelling | mit-1721.1/468202019-04-10T11:56:26Z Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution Roy, Nicholas He, Ruijie Nicholas Roy Robotics, Vision & Sensor Networks Online, forward-search techniques have demonstrated promising results for solving problems in partially observable environments. These techniques depend on the ability to efficiently search and evaluate the set of beliefs reachable from the current belief. However, enumerating or sampling action-observation sequences to compute the reachable beliefs is computationally demanding; coupled with the need to satisfy real-time constraints, existing online solvers can only search to a limited depth. In this paper, we propose that policies can be generated directly from the distribution of the agent's posterior belief. When the underlying state distribution is Gaussian, and the observation function is an exponential family distribution, we can calculate this distribution of beliefs without enumerating the possible observations. This property not only enables us to plan in problems with large observation spaces, but also allows us to search deeper by considering policies composed of multi-step action sequences. We present the Posterior Belief Distribution (PBD) algorithm, an efficient forward-search POMDP planner for continuous domains, demonstrating that better policies are generated when we can perform deeper forward search. 2009-09-28T21:00:15Z 2009-09-28T21:00:15Z 2009-09-23 http://hdl.handle.net/1721.1/46820 MIT-CSAIL-TR-2009-044 Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported http://creativecommons.org/licenses/by-nc-nd/3.0/ 12 p. application/pdf application/postscript |
| spellingShingle | Roy, Nicholas He, Ruijie Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution |
| title | Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution |
| title_full | Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution |
| title_fullStr | Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution |
| title_full_unstemmed | Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution |
| title_short | Efficient POMDP Forward Search by Predicting the Posterior Belief Distribution |
| title_sort | efficient pomdp forward search by predicting the posterior belief distribution |
| url | http://hdl.handle.net/1721.1/46820 |
| work_keys_str_mv | AT roynicholas efficientpomdpforwardsearchbypredictingtheposteriorbeliefdistribution AT heruijie efficientpomdpforwardsearchbypredictingtheposteriorbeliefdistribution |