Reinforcement Learning for Mapping Instructions to Actions
In this paper, we present a reinforcement learning approach for mapping natural language instructions to sequences of executable actions. We assume access to a reward function that defines the quality of the executed actions. During training, the learner repeatedly constructs action sequences for a...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computational Linguistics
2010
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/59313 https://orcid.org/0000-0002-2921-8201 |
| _version_ | 1811070718361731072 |
|---|---|
| author | Branavan, Satchuthanan R. Chen, Harr Zettlemoyer, Luke S. Barzilay, Regina |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Branavan, Satchuthanan R. Chen, Harr Zettlemoyer, Luke S. Barzilay, Regina |
| author_sort | Branavan, Satchuthanan R. |
| collection | MIT |
| description | In this paper, we present a reinforcement learning approach for mapping natural language instructions to sequences of executable actions. We assume access to a reward function that defines the quality of the executed actions. During training, the learner repeatedly constructs action sequences for a set of documents, executes those actions, and observes the resulting reward. We use a policy gradient algorithm to estimate the parameters of a log-linear model for action selection. We apply our method to interpret instructions in two domains --- Windows troubleshooting guides and game tutorials. Our results demonstrate that this method can rival supervised learning techniques while requiring few or no annotated training examples. |
| first_indexed | 2024-09-23T08:40:28Z |
| format | Article |
| id | mit-1721.1/59313 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:40:28Z |
| publishDate | 2010 |
| publisher | Association for Computational Linguistics |
| record_format | dspace |
| spelling | mit-1721.1/593132022-09-30T10:26:12Z Reinforcement Learning for Mapping Instructions to Actions Branavan, Satchuthanan R. Chen, Harr Zettlemoyer, Luke S. Barzilay, Regina Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Barzilay, Regina Branavan, Satchuthanan R. Chen, Harr Zettlemoyer, Luke S. Barzilay, Regina algorithms design experimentation languages measurement performance In this paper, we present a reinforcement learning approach for mapping natural language instructions to sequences of executable actions. We assume access to a reward function that defines the quality of the executed actions. During training, the learner repeatedly constructs action sequences for a set of documents, executes those actions, and observes the resulting reward. We use a policy gradient algorithm to estimate the parameters of a log-linear model for action selection. We apply our method to interpret instructions in two domains --- Windows troubleshooting guides and game tutorials. Our results demonstrate that this method can rival supervised learning techniques while requiring few or no annotated training examples. National Science Foundation (U.S.) (grant IIS-0448168) National Science Foundation (U.S.) (grant IIS-0835445) United States. Office of Naval Research National Science Foundation (U.S.) (grant IIS-0835652) 2010-10-14T12:46:32Z 2010-10-14T12:46:32Z 2009-08 2009-08 Article http://purl.org/eprint/type/ConferencePaper 978-1-932432-45-9 http://hdl.handle.net/1721.1/59313 Branavan, S.R.K., Harr Chen, Luke S. Zettlemoyer, and Regina Barzilay (2009). "Reinforcement learning for mapping instructions to actions." Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (Morristown, N.J.: Association for Computational Linguistics): 82-90. © Association for Computing Machinery. https://orcid.org/0000-0002-2921-8201 en_US http://portal.acm.org/citation.cfm?id=1687892 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP Attribution-Noncommercial-Share Alike 3.0 Unported http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Association for Computational Linguistics MIT web domain |
| spellingShingle | algorithms design experimentation languages measurement performance Branavan, Satchuthanan R. Chen, Harr Zettlemoyer, Luke S. Barzilay, Regina Reinforcement Learning for Mapping Instructions to Actions |
| title | Reinforcement Learning for Mapping Instructions to Actions |
| title_full | Reinforcement Learning for Mapping Instructions to Actions |
| title_fullStr | Reinforcement Learning for Mapping Instructions to Actions |
| title_full_unstemmed | Reinforcement Learning for Mapping Instructions to Actions |
| title_short | Reinforcement Learning for Mapping Instructions to Actions |
| title_sort | reinforcement learning for mapping instructions to actions |
| topic | algorithms design experimentation languages measurement performance |
| url | http://hdl.handle.net/1721.1/59313 https://orcid.org/0000-0002-2921-8201 |
| work_keys_str_mv | AT branavansatchuthananr reinforcementlearningformappinginstructionstoactions AT chenharr reinforcementlearningformappinginstructionstoactions AT zettlemoyerlukes reinforcementlearningformappinginstructionstoactions AT barzilayregina reinforcementlearningformappinginstructionstoactions |