Neural controlled differential equations for irregular time series
Neural ordinary differential equations are an attractive option for modelling temporal dynamics. However, a fundamental issue is that the solution to an ordinary differential equation is determined by its initial condition, and there is no mechanism for adjusting the trajectory based on subsequent o...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Neural Information Processing Systems Foundation, Inc.
2020
|
| _version_ | 1797087737568821248 |
|---|---|
| author | Kidger, P Morrill, J Foster, J Lyons, T |
| author_facet | Kidger, P Morrill, J Foster, J Lyons, T |
| author_sort | Kidger, P |
| collection | OXFORD |
| description | Neural ordinary differential equations are an attractive option for modelling temporal dynamics. However, a fundamental issue is that the solution to an ordinary differential equation is determined by its initial condition, and there is no mechanism for adjusting the trajectory based on subsequent observations. Here, we demonstrate how this may be resolved through the well-understood mathematics of controlled differential equations. The resulting neural controlled differential equation model is directly applicable to the general setting of partially-observed irregularly-sampled multivariate time series, and (unlike previous work on this problem) it may utilise memory-efficient adjoint-based backpropagation even across observations. We demonstrate that our model achieves state-of-the-art performance against similar (ODE or RNN based) models in empirical studies on a range of datasets. Finally we provide theoretical results demonstrating universal approximation, and that our model subsumes alternative ODE models. |
| first_indexed | 2024-03-07T02:39:57Z |
| format | Conference item |
| id | oxford-uuid:aa145098-c206-4729-b0e7-1123b6f2b416 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T02:39:57Z |
| publishDate | 2020 |
| publisher | Neural Information Processing Systems Foundation, Inc. |
| record_format | dspace |
| spelling | oxford-uuid:aa145098-c206-4729-b0e7-1123b6f2b4162022-03-27T03:12:52ZNeural controlled differential equations for irregular time seriesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:aa145098-c206-4729-b0e7-1123b6f2b416EnglishSymplectic ElementsNeural Information Processing Systems Foundation, Inc.2020Kidger, PMorrill, JFoster, JLyons, TNeural ordinary differential equations are an attractive option for modelling temporal dynamics. However, a fundamental issue is that the solution to an ordinary differential equation is determined by its initial condition, and there is no mechanism for adjusting the trajectory based on subsequent observations. Here, we demonstrate how this may be resolved through the well-understood mathematics of controlled differential equations. The resulting neural controlled differential equation model is directly applicable to the general setting of partially-observed irregularly-sampled multivariate time series, and (unlike previous work on this problem) it may utilise memory-efficient adjoint-based backpropagation even across observations. We demonstrate that our model achieves state-of-the-art performance against similar (ODE or RNN based) models in empirical studies on a range of datasets. Finally we provide theoretical results demonstrating universal approximation, and that our model subsumes alternative ODE models. |
| spellingShingle | Kidger, P Morrill, J Foster, J Lyons, T Neural controlled differential equations for irregular time series |
| title | Neural controlled differential equations for irregular time series |
| title_full | Neural controlled differential equations for irregular time series |
| title_fullStr | Neural controlled differential equations for irregular time series |
| title_full_unstemmed | Neural controlled differential equations for irregular time series |
| title_short | Neural controlled differential equations for irregular time series |
| title_sort | neural controlled differential equations for irregular time series |
| work_keys_str_mv | AT kidgerp neuralcontrolleddifferentialequationsforirregulartimeseries AT morrillj neuralcontrolleddifferentialequationsforirregulartimeseries AT fosterj neuralcontrolleddifferentialequationsforirregulartimeseries AT lyonst neuralcontrolleddifferentialequationsforirregulartimeseries |