Planning with Uncertain Specifications (PUnS)
© 2016 IEEE. Reward engineering is crucial to high performance in reinforcement learning systems. Prior research into reward design has largely focused on Markovian functions representing the reward. While there has been research into expressing non-Markov rewards as linear temporal logic (LTL) form...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135354 |
| _version_ | 1826197039420014592 |
|---|---|
| author | Shah, Ankit Li, Shen Shah, Julie |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Shah, Ankit Li, Shen Shah, Julie |
| author_sort | Shah, Ankit |
| collection | MIT |
| description | © 2016 IEEE. Reward engineering is crucial to high performance in reinforcement learning systems. Prior research into reward design has largely focused on Markovian functions representing the reward. While there has been research into expressing non-Markov rewards as linear temporal logic (LTL) formulas, this has focused on task specifications directly defined by the user. However, in many real-world applications, task specifications are ambiguous, and can only be expressed as a belief over LTL formulas. In this letter, we introduce planning with uncertain specifications (PUnS), a novel formulation that addresses the challenge posed by non-Markovian specifications expressed as beliefs over LTL formulas. We present four criteria that capture the semantics of satisfying a belief over specifications for different applications, and analyze the qualitative implications of these criteria within a synthetic domain. We demonstrate the existence of an equivalent Markov decision process (MDP) for any instance of PUnS. Finally, we demonstrate our approach on the real-world task of setting a dinner table automatically with a robot that inferred task specifications from human demonstrations. |
| first_indexed | 2024-09-23T10:41:30Z |
| format | Article |
| id | mit-1721.1/135354 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T10:41:30Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1353542023-02-24T18:20:59Z Planning with Uncertain Specifications (PUnS) Shah, Ankit Li, Shen Shah, Julie Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics © 2016 IEEE. Reward engineering is crucial to high performance in reinforcement learning systems. Prior research into reward design has largely focused on Markovian functions representing the reward. While there has been research into expressing non-Markov rewards as linear temporal logic (LTL) formulas, this has focused on task specifications directly defined by the user. However, in many real-world applications, task specifications are ambiguous, and can only be expressed as a belief over LTL formulas. In this letter, we introduce planning with uncertain specifications (PUnS), a novel formulation that addresses the challenge posed by non-Markovian specifications expressed as beliefs over LTL formulas. We present four criteria that capture the semantics of satisfying a belief over specifications for different applications, and analyze the qualitative implications of these criteria within a synthetic domain. We demonstrate the existence of an equivalent Markov decision process (MDP) for any instance of PUnS. Finally, we demonstrate our approach on the real-world task of setting a dinner table automatically with a robot that inferred task specifications from human demonstrations. 2021-10-27T20:23:06Z 2021-10-27T20:23:06Z 2020 2021-05-04T14:59:25Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135354 en 10.1109/LRA.2020.2977217 IEEE Robotics and Automation Letters Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Shah, Ankit Li, Shen Shah, Julie Planning with Uncertain Specifications (PUnS) |
| title | Planning with Uncertain Specifications (PUnS) |
| title_full | Planning with Uncertain Specifications (PUnS) |
| title_fullStr | Planning with Uncertain Specifications (PUnS) |
| title_full_unstemmed | Planning with Uncertain Specifications (PUnS) |
| title_short | Planning with Uncertain Specifications (PUnS) |
| title_sort | planning with uncertain specifications puns |
| url | https://hdl.handle.net/1721.1/135354 |
| work_keys_str_mv | AT shahankit planningwithuncertainspecificationspuns AT lishen planningwithuncertainspecificationspuns AT shahjulie planningwithuncertainspecificationspuns |