Sampling-based algorithms for continuous-time POMDPs
This paper focuses on a continuous-time, continuous-space formulation of the stochastic optimal control problem with nonlinear dynamics and observation noise. We lay the mathematical foundations to construct, via incremental sampling, an approximating sequence of discrete-time finite-state partially...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2013
|
| Online Access: | http://hdl.handle.net/1721.1/81825 https://orcid.org/0000-0002-0505-1400 https://orcid.org/0000-0002-2225-7275 |
| _version_ | 1811092796959883264 |
|---|---|
| author | Chaudhari, Pratik Anil Karaman, Sertac Hsu, David Frazzoli, Emilio |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Chaudhari, Pratik Anil Karaman, Sertac Hsu, David Frazzoli, Emilio |
| author_sort | Chaudhari, Pratik Anil |
| collection | MIT |
| description | This paper focuses on a continuous-time, continuous-space formulation of the stochastic optimal control problem with nonlinear dynamics and observation noise. We lay the mathematical foundations to construct, via incremental sampling, an approximating sequence of discrete-time finite-state partially observable Markov decision processes (POMDPs), such that the behavior of successive approximations converges to the behavior of the original continuous system in an appropriate sense. We also show that the optimal cost function and control policies for these POMDP approximations converge almost surely to their counterparts for the underlying continuous system in the limit. We demonstrate this approach on two popular continuous-time problems, viz., the Linear-Quadratic-Gaussian (LQG) control problem and the light-dark domain problem. |
| first_indexed | 2024-09-23T15:26:35Z |
| format | Article |
| id | mit-1721.1/81825 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:26:35Z |
| publishDate | 2013 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/818252022-09-29T14:41:38Z Sampling-based algorithms for continuous-time POMDPs Chaudhari, Pratik Anil Karaman, Sertac Hsu, David Frazzoli, Emilio Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Chaudhari, Pratik Anil Karaman, Sertac Frazzoli, Emilio This paper focuses on a continuous-time, continuous-space formulation of the stochastic optimal control problem with nonlinear dynamics and observation noise. We lay the mathematical foundations to construct, via incremental sampling, an approximating sequence of discrete-time finite-state partially observable Markov decision processes (POMDPs), such that the behavior of successive approximations converges to the behavior of the original continuous system in an appropriate sense. We also show that the optimal cost function and control policies for these POMDP approximations converge almost surely to their counterparts for the underlying continuous system in the limit. We demonstrate this approach on two popular continuous-time problems, viz., the Linear-Quadratic-Gaussian (LQG) control problem and the light-dark domain problem. United States. Army Research Office. Multidisciplinary University Research Initiative (Grant W911NF-11-1-0046) 2013-10-29T13:48:44Z 2013-10-29T13:48:44Z 2013-06 Article http://purl.org/eprint/type/ConferencePaper 978-1-4799-0177-7 http://hdl.handle.net/1721.1/81825 Chaudhari, Pratik Anil et al. "Sampling-based algorithms for continuous-time POMDPs." IEEE American Control Conference (ACC), 2013. https://orcid.org/0000-0002-0505-1400 https://orcid.org/0000-0002-2225-7275 en_US http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6580549 Proceedings of the 2013 American Control Conference (ACC) Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Chaudhari, Pratik Anil Karaman, Sertac Hsu, David Frazzoli, Emilio Sampling-based algorithms for continuous-time POMDPs |
| title | Sampling-based algorithms for continuous-time POMDPs |
| title_full | Sampling-based algorithms for continuous-time POMDPs |
| title_fullStr | Sampling-based algorithms for continuous-time POMDPs |
| title_full_unstemmed | Sampling-based algorithms for continuous-time POMDPs |
| title_short | Sampling-based algorithms for continuous-time POMDPs |
| title_sort | sampling based algorithms for continuous time pomdps |
| url | http://hdl.handle.net/1721.1/81825 https://orcid.org/0000-0002-0505-1400 https://orcid.org/0000-0002-2225-7275 |
| work_keys_str_mv | AT chaudharipratikanil samplingbasedalgorithmsforcontinuoustimepomdps AT karamansertac samplingbasedalgorithmsforcontinuoustimepomdps AT hsudavid samplingbasedalgorithmsforcontinuoustimepomdps AT frazzoliemilio samplingbasedalgorithmsforcontinuoustimepomdps |