Modeling Task Uncertainty for Safe Meta-Imitation Learning
To endow robots with the flexibility to perform a wide range of tasks in diverse and complex environments, learning their controller from experience data is a promising approach. In particular, some recent meta-learning methods are shown to solve novel tasks by leveraging their experience of perform...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2020-11-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frobt.2020.606361/full |
| _version_ | 1818928787381813248 |
|---|---|
| author | Tatsuya Matsushima Naruya Kondo Yusuke Iwasawa Kaoru Nasuno Kaoru Nasuno Yutaka Matsuo |
| author_facet | Tatsuya Matsushima Naruya Kondo Yusuke Iwasawa Kaoru Nasuno Kaoru Nasuno Yutaka Matsuo |
| author_sort | Tatsuya Matsushima |
| collection | DOAJ |
| description | To endow robots with the flexibility to perform a wide range of tasks in diverse and complex environments, learning their controller from experience data is a promising approach. In particular, some recent meta-learning methods are shown to solve novel tasks by leveraging their experience of performing other tasks during training. Although studies around meta-learning of robot control have worked on improving the performance, the safety issue has not been fully explored, which is also an important consideration in the deployment. In this paper, we firstly relate uncertainty on task inference with the safety in meta-learning of visual imitation, and then propose a novel framework for estimating the task uncertainty through probabilistic inference in the task-embedding space, called PETNet. We validate PETNet with a manipulation task with a simulated robot arm in terms of the task performance and uncertainty evaluation on task inference. Following the standard benchmark procedure in meta-imitation learning, we show PETNet can achieve the same or higher level of performance (success rate of novel tasks at meta-test time) as previous methods. In addition, by testing PETNet with semantically inappropriate or synthesized out-of-distribution demonstrations, PETNet shows the ability to capture the uncertainty about the tasks inherent in the given demonstrations, which allows the robot to identify situations where the controller might not perform properly. These results illustrate our proposal takes a significant step forward to the safe deployment of robot learning systems into diverse tasks and environments. |
| first_indexed | 2024-12-20T03:34:27Z |
| format | Article |
| id | doaj.art-12e4e58ea9b94e5398dc3798d531a6df |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-12-20T03:34:27Z |
| publishDate | 2020-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-12e4e58ea9b94e5398dc3798d531a6df2022-12-21T19:54:55ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442020-11-01710.3389/frobt.2020.606361606361Modeling Task Uncertainty for Safe Meta-Imitation LearningTatsuya Matsushima0Naruya Kondo1Yusuke Iwasawa2Kaoru Nasuno3Kaoru Nasuno4Yutaka Matsuo5School of Engineering, The University of Tokyo, Tokyo, JapanSchool of Engineering, The University of Tokyo, Tokyo, JapanSchool of Engineering, The University of Tokyo, Tokyo, JapanSchool of Engineering, The University of Tokyo, Tokyo, JapanDeepX Inc., Tokyo, JapanSchool of Engineering, The University of Tokyo, Tokyo, JapanTo endow robots with the flexibility to perform a wide range of tasks in diverse and complex environments, learning their controller from experience data is a promising approach. In particular, some recent meta-learning methods are shown to solve novel tasks by leveraging their experience of performing other tasks during training. Although studies around meta-learning of robot control have worked on improving the performance, the safety issue has not been fully explored, which is also an important consideration in the deployment. In this paper, we firstly relate uncertainty on task inference with the safety in meta-learning of visual imitation, and then propose a novel framework for estimating the task uncertainty through probabilistic inference in the task-embedding space, called PETNet. We validate PETNet with a manipulation task with a simulated robot arm in terms of the task performance and uncertainty evaluation on task inference. Following the standard benchmark procedure in meta-imitation learning, we show PETNet can achieve the same or higher level of performance (success rate of novel tasks at meta-test time) as previous methods. In addition, by testing PETNet with semantically inappropriate or synthesized out-of-distribution demonstrations, PETNet shows the ability to capture the uncertainty about the tasks inherent in the given demonstrations, which allows the robot to identify situations where the controller might not perform properly. These results illustrate our proposal takes a significant step forward to the safe deployment of robot learning systems into diverse tasks and environments.https://www.frontiersin.org/articles/10.3389/frobt.2020.606361/fullmeta-learningimitation learningrobot learningtask uncertaintysafety |
| spellingShingle | Tatsuya Matsushima Naruya Kondo Yusuke Iwasawa Kaoru Nasuno Kaoru Nasuno Yutaka Matsuo Modeling Task Uncertainty for Safe Meta-Imitation Learning Frontiers in Robotics and AI meta-learning imitation learning robot learning task uncertainty safety |
| title | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
| title_full | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
| title_fullStr | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
| title_full_unstemmed | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
| title_short | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
| title_sort | modeling task uncertainty for safe meta imitation learning |
| topic | meta-learning imitation learning robot learning task uncertainty safety |
| url | https://www.frontiersin.org/articles/10.3389/frobt.2020.606361/full |
| work_keys_str_mv | AT tatsuyamatsushima modelingtaskuncertaintyforsafemetaimitationlearning AT naruyakondo modelingtaskuncertaintyforsafemetaimitationlearning AT yusukeiwasawa modelingtaskuncertaintyforsafemetaimitationlearning AT kaorunasuno modelingtaskuncertaintyforsafemetaimitationlearning AT kaorunasuno modelingtaskuncertaintyforsafemetaimitationlearning AT yutakamatsuo modelingtaskuncertaintyforsafemetaimitationlearning |