Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2017
|
| Online Access: | http://hdl.handle.net/1721.1/107887 https://orcid.org/0000-0003-1338-8107 https://orcid.org/0000-0001-8239-5963 |
| _version_ | 1811086140413837312 |
|---|---|
| author | Shah, Julie A Nikolaidis, Stefanos Ramakrishnan, Ramya Gu, Keren |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Shah, Julie A Nikolaidis, Stefanos Ramakrishnan, Ramya Gu, Keren |
| author_sort | Shah, Julie A |
| collection | MIT |
| description | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p<0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
| first_indexed | 2024-09-23T13:21:29Z |
| format | Article |
| id | mit-1721.1/107887 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T13:21:29Z |
| publishDate | 2017 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1078872022-09-28T13:39:21Z Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks Shah, Julie A Nikolaidis, Stefanos Ramakrishnan, Ramya Gu, Keren Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Shah, Julie A Nikolaidis, Stefanos Ramakrishnan, Ramya Gu, Keren We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p<0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. 2017-04-05T20:03:20Z 2017-04-05T20:03:20Z 2015-03 Article http://purl.org/eprint/type/ConferencePaper 9781450328838 http://hdl.handle.net/1721.1/107887 Nikolaidis, Stefanos et al. “Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks.” ACM Press, 2015. 189–196. https://orcid.org/0000-0003-1338-8107 https://orcid.org/0000-0001-8239-5963 en_US http://dx.doi.org/10.1145/2696454.2696455 Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction - HRI '15 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Shah, Julie A Nikolaidis, Stefanos Ramakrishnan, Ramya Gu, Keren Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| title | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| title_full | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| title_fullStr | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| title_full_unstemmed | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| title_short | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| title_sort | efficient model learning from joint action demonstrations for human robot collaborative tasks |
| url | http://hdl.handle.net/1721.1/107887 https://orcid.org/0000-0003-1338-8107 https://orcid.org/0000-0001-8239-5963 |
| work_keys_str_mv | AT shahjuliea efficientmodellearningfromjointactiondemonstrationsforhumanrobotcollaborativetasks AT nikolaidisstefanos efficientmodellearningfromjointactiondemonstrationsforhumanrobotcollaborativetasks AT ramakrishnanramya efficientmodellearningfromjointactiondemonstrationsforhumanrobotcollaborativetasks AT gukeren efficientmodellearningfromjointactiondemonstrationsforhumanrobotcollaborativetasks |