Learn what matters: cross-domain imitation learning with task-relevant embeddings
We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a sca...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Curran Associates
2023
|
| _version_ | 1797112955619246080 |
|---|---|
| author | Franzmeyer, T Torr, PHS Henriques, JF |
| author_facet | Franzmeyer, T Torr, PHS Henriques, JF |
| author_sort | Franzmeyer, T |
| collection | OXFORD |
| description | We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge. We jointly train the learner agent's policy and learn a mapping between the learner and expert domains with adversarial training. We effect this by using a mutual information criterion to find an embedding of the expert's state space that contains task-relevant information and is invariant to domain specifics. This step significantly simplifies estimating the mapping between the learner and expert domains and hence facilitates end-to-end learning. We demonstrate successful transfer of policies between considerably different domains, without extra supervision such as additional demonstrations, and in situations where other methods fail. |
| first_indexed | 2024-03-07T08:29:46Z |
| format | Conference item |
| id | oxford-uuid:d1dbb71a-5d3f-40fc-aa64-5ef120569894 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-04-09T03:55:23Z |
| publishDate | 2023 |
| publisher | Curran Associates |
| record_format | dspace |
| spelling | oxford-uuid:d1dbb71a-5d3f-40fc-aa64-5ef1205698942024-03-12T10:45:50ZLearn what matters: cross-domain imitation learning with task-relevant embeddingsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:d1dbb71a-5d3f-40fc-aa64-5ef120569894EnglishSymplectic ElementsCurran Associates2023Franzmeyer, TTorr, PHSHenriques, JFWe study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge. We jointly train the learner agent's policy and learn a mapping between the learner and expert domains with adversarial training. We effect this by using a mutual information criterion to find an embedding of the expert's state space that contains task-relevant information and is invariant to domain specifics. This step significantly simplifies estimating the mapping between the learner and expert domains and hence facilitates end-to-end learning. We demonstrate successful transfer of policies between considerably different domains, without extra supervision such as additional demonstrations, and in situations where other methods fail. |
| spellingShingle | Franzmeyer, T Torr, PHS Henriques, JF Learn what matters: cross-domain imitation learning with task-relevant embeddings |
| title | Learn what matters: cross-domain imitation learning with task-relevant embeddings |
| title_full | Learn what matters: cross-domain imitation learning with task-relevant embeddings |
| title_fullStr | Learn what matters: cross-domain imitation learning with task-relevant embeddings |
| title_full_unstemmed | Learn what matters: cross-domain imitation learning with task-relevant embeddings |
| title_short | Learn what matters: cross-domain imitation learning with task-relevant embeddings |
| title_sort | learn what matters cross domain imitation learning with task relevant embeddings |
| work_keys_str_mv | AT franzmeyert learnwhatmatterscrossdomainimitationlearningwithtaskrelevantembeddings AT torrphs learnwhatmatterscrossdomainimitationlearningwithtaskrelevantembeddings AT henriquesjf learnwhatmatterscrossdomainimitationlearningwithtaskrelevantembeddings |