SigGPDE: scaling sparse Gaussian Processes on sequential data
Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contrib...
| Main Authors: | , , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Journal of Machine Learning Research
2021
|
| _version_ | 1826312582489702400 |
|---|---|
| author | Lemercier, M Salvi, C Cass, T Bonilla, EV Damonlas, T Lyons, T |
| author_facet | Lemercier, M Salvi, C Cass, T Bonilla, EV Damonlas, T Lyons, T |
| author_sort | Lemercier, M |
| collection | OXFORD |
| description | Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series. |
| first_indexed | 2024-03-07T08:25:08Z |
| format | Conference item |
| id | oxford-uuid:b5454fac-07d1-4482-b155-fd502d5271e7 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-04-09T03:57:03Z |
| publishDate | 2021 |
| publisher | Journal of Machine Learning Research |
| record_format | dspace |
| spelling | oxford-uuid:b5454fac-07d1-4482-b155-fd502d5271e72024-03-21T16:08:43ZSigGPDE: scaling sparse Gaussian Processes on sequential dataConference itemhttp://purl.org/coar/resource_type/c_5794uuid:b5454fac-07d1-4482-b155-fd502d5271e7EnglishSymplectic ElementsJournal of Machine Learning Research2021Lemercier, MSalvi, CCass, TBonilla, EVDamonlas, TLyons, TMaking predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series. |
| spellingShingle | Lemercier, M Salvi, C Cass, T Bonilla, EV Damonlas, T Lyons, T SigGPDE: scaling sparse Gaussian Processes on sequential data |
| title | SigGPDE: scaling sparse Gaussian Processes on sequential data |
| title_full | SigGPDE: scaling sparse Gaussian Processes on sequential data |
| title_fullStr | SigGPDE: scaling sparse Gaussian Processes on sequential data |
| title_full_unstemmed | SigGPDE: scaling sparse Gaussian Processes on sequential data |
| title_short | SigGPDE: scaling sparse Gaussian Processes on sequential data |
| title_sort | siggpde scaling sparse gaussian processes on sequential data |
| work_keys_str_mv | AT lemercierm siggpdescalingsparsegaussianprocessesonsequentialdata AT salvic siggpdescalingsparsegaussianprocessesonsequentialdata AT casst siggpdescalingsparsegaussianprocessesonsequentialdata AT bonillaev siggpdescalingsparsegaussianprocessesonsequentialdata AT damonlast siggpdescalingsparsegaussianprocessesonsequentialdata AT lyonst siggpdescalingsparsegaussianprocessesonsequentialdata |