Learning models of sequential decision-making with partial specification of agent behavior
Artificial agents that interact with other (human or artificial) agents require models in order to reason about those other agents’ behavior. In addition to the predictive utility of these models, maintaining a model that is aligned with an agent’s true generative model of behavior is critical for e...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
2020
|
| Online Access: | https://hdl.handle.net/1721.1/125889 |
| _version_ | 1811094034243911680 |
|---|---|
| author | Unhelkar, Vaibhav Vasant Shah, Julie A |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Unhelkar, Vaibhav Vasant Shah, Julie A |
| author_sort | Unhelkar, Vaibhav Vasant |
| collection | MIT |
| description | Artificial agents that interact with other (human or artificial) agents require models in order to reason about those other agents’ behavior. In addition to the predictive utility of these models, maintaining a model that is aligned with an agent’s true generative model of behavior is critical for effective human-agent interaction. In applications wherein observations and partial specification of the agent’s behavior are available, achieving model alignment is challenging for a variety of reasons. For one, the agent’s decision factors are often not completely known; further, prior approaches that rely upon observations of agents’ behavior alone can fail to recover the true model, since multiple models can explain observed behavior equally well. To achieve better model alignment, we provide a novel approach capable of learning aligned models that conform to partial knowledge of the agent’s behavior. Central to our approach are a factored model of behavior (AMM), along with Bayesian nonparametric priors, and an inference approach capable of incorporating partial specifications as constraints for model learning. We evaluate our approach in experiments and demonstrate improvements in metrics of model alignment. |
| first_indexed | 2024-09-23T15:54:27Z |
| format | Article |
| id | mit-1721.1/125889 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T15:54:27Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | mit-1721.1/1258892022-09-29T16:57:42Z Learning models of sequential decision-making with partial specification of agent behavior Unhelkar, Vaibhav Vasant Shah, Julie A Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Artificial agents that interact with other (human or artificial) agents require models in order to reason about those other agents’ behavior. In addition to the predictive utility of these models, maintaining a model that is aligned with an agent’s true generative model of behavior is critical for effective human-agent interaction. In applications wherein observations and partial specification of the agent’s behavior are available, achieving model alignment is challenging for a variety of reasons. For one, the agent’s decision factors are often not completely known; further, prior approaches that rely upon observations of agents’ behavior alone can fail to recover the true model, since multiple models can explain observed behavior equally well. To achieve better model alignment, we provide a novel approach capable of learning aligned models that conform to partial knowledge of the agent’s behavior. Central to our approach are a factored model of behavior (AMM), along with Bayesian nonparametric priors, and an inference approach capable of incorporating partial specifications as constraints for model learning. We evaluate our approach in experiments and demonstrate improvements in metrics of model alignment. 2020-06-19T17:59:13Z 2020-06-19T17:59:13Z 2019 2019-11-01T12:53:18Z Article http://purl.org/eprint/type/ConferencePaper 2374-3468 https://hdl.handle.net/1721.1/125889 Unhelkar, Vaibhav V., and Julie A. Shah, "Learning models of sequential decision-making with partial specification of agent behavior." Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Jan. 27–Feb. 1, 2019, Honolulu, Hawai'i, AAAI Press, 2019: doi 10.1609/aaai.v33i01.33012522 ©2019 Author(s) en 10.1609/aaai.v33i01.33012522 Proceedings of the AAAI Conference on Artificial Intelligence Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf MIT web domain |
| spellingShingle | Unhelkar, Vaibhav Vasant Shah, Julie A Learning models of sequential decision-making with partial specification of agent behavior |
| title | Learning models of sequential decision-making with partial specification of agent behavior |
| title_full | Learning models of sequential decision-making with partial specification of agent behavior |
| title_fullStr | Learning models of sequential decision-making with partial specification of agent behavior |
| title_full_unstemmed | Learning models of sequential decision-making with partial specification of agent behavior |
| title_short | Learning models of sequential decision-making with partial specification of agent behavior |
| title_sort | learning models of sequential decision making with partial specification of agent behavior |
| url | https://hdl.handle.net/1721.1/125889 |
| work_keys_str_mv | AT unhelkarvaibhavvasant learningmodelsofsequentialdecisionmakingwithpartialspecificationofagentbehavior AT shahjuliea learningmodelsofsequentialdecisionmakingwithpartialspecificationofagentbehavior |