Alternating optimisation and quadrature for robust control
Bayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators does not utilise the opportunity to improve learning by adjusting certain environment variables: state features that are...
| Main Authors: | , , , , , |
|---|---|
| Format: | Conference item |
| Published: |
AAAI Press
2018
|
| _version_ | 1797088112660185088 |
|---|---|
| author | Paul, S Chatzilygeroudis, K Ciosek, K Mouret, J Osborne, M Whiteson, S |
| author_facet | Paul, S Chatzilygeroudis, K Ciosek, K Mouret, J Osborne, M Whiteson, S |
| author_sort | Paul, S |
| collection | OXFORD |
| description | Bayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators does not utilise the opportunity to improve learning by adjusting certain environment variables: state features that are unobservable and randomly determined by the environment in a physical setting but are controllable in a simulator. This paper considers the problem of finding a robust policy while taking into account the impact of environment variables. We present Alternating Optimisation and Quadrature (ALOQ), which uses Bayesian optimisation and Bayesian quadrature to address such settings. ALOQ is robust to the presence of significant rare events, which may not be observable under random sampling, but play a substantial role in determining the optimal policy. Experimental results across different domains show that ALOQ can learn more efficiently and robustly than existing methods. |
| first_indexed | 2024-03-07T02:45:15Z |
| format | Conference item |
| id | oxford-uuid:abd7c997-b0fb-4e66-b601-f82184500cbf |
| institution | University of Oxford |
| last_indexed | 2024-03-07T02:45:15Z |
| publishDate | 2018 |
| publisher | AAAI Press |
| record_format | dspace |
| spelling | oxford-uuid:abd7c997-b0fb-4e66-b601-f82184500cbf2022-03-27T03:24:42ZAlternating optimisation and quadrature for robust controlConference itemhttp://purl.org/coar/resource_type/c_5794uuid:abd7c997-b0fb-4e66-b601-f82184500cbfSymplectic Elements at OxfordAAAI Press2018Paul, SChatzilygeroudis, KCiosek, KMouret, JOsborne, MWhiteson, SBayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators does not utilise the opportunity to improve learning by adjusting certain environment variables: state features that are unobservable and randomly determined by the environment in a physical setting but are controllable in a simulator. This paper considers the problem of finding a robust policy while taking into account the impact of environment variables. We present Alternating Optimisation and Quadrature (ALOQ), which uses Bayesian optimisation and Bayesian quadrature to address such settings. ALOQ is robust to the presence of significant rare events, which may not be observable under random sampling, but play a substantial role in determining the optimal policy. Experimental results across different domains show that ALOQ can learn more efficiently and robustly than existing methods. |
| spellingShingle | Paul, S Chatzilygeroudis, K Ciosek, K Mouret, J Osborne, M Whiteson, S Alternating optimisation and quadrature for robust control |
| title | Alternating optimisation and quadrature for robust control |
| title_full | Alternating optimisation and quadrature for robust control |
| title_fullStr | Alternating optimisation and quadrature for robust control |
| title_full_unstemmed | Alternating optimisation and quadrature for robust control |
| title_short | Alternating optimisation and quadrature for robust control |
| title_sort | alternating optimisation and quadrature for robust control |
| work_keys_str_mv | AT pauls alternatingoptimisationandquadratureforrobustcontrol AT chatzilygeroudisk alternatingoptimisationandquadratureforrobustcontrol AT ciosekk alternatingoptimisationandquadratureforrobustcontrol AT mouretj alternatingoptimisationandquadratureforrobustcontrol AT osbornem alternatingoptimisationandquadratureforrobustcontrol AT whitesons alternatingoptimisationandquadratureforrobustcontrol |