Neural controlled differential equations for online prediction tasks
Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations r...
| Main Authors: | , , , |
|---|---|
| Format: | Internet publication |
| Language: | English |
| Published: |
2021
|
| _version_ | 1797109904294543360 |
|---|---|
| author | Morrill, J Kidger, P Yang, L Lyons, T |
| author_facet | Morrill, J Kidger, P Yang, L Lyons, T |
| author_sort | Morrill, J |
| collection | OXFORD |
| description | Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in \textit{online prediction tasks}, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that interpolation schemes for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks. |
| first_indexed | 2024-03-07T07:47:41Z |
| format | Internet publication |
| id | oxford-uuid:e8f04ac4-510a-4800-8905-ad38ac809061 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:47:41Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | oxford-uuid:e8f04ac4-510a-4800-8905-ad38ac8090612023-06-09T12:32:50ZNeural controlled differential equations for online prediction tasksInternet publicationhttp://purl.org/coar/resource_type/c_7ad9uuid:e8f04ac4-510a-4800-8905-ad38ac809061EnglishSymplectic Elements2021Morrill, JKidger, PYang, LLyons, TNeural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in \textit{online prediction tasks}, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that interpolation schemes for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks. |
| spellingShingle | Morrill, J Kidger, P Yang, L Lyons, T Neural controlled differential equations for online prediction tasks |
| title | Neural controlled differential equations for online prediction tasks |
| title_full | Neural controlled differential equations for online prediction tasks |
| title_fullStr | Neural controlled differential equations for online prediction tasks |
| title_full_unstemmed | Neural controlled differential equations for online prediction tasks |
| title_short | Neural controlled differential equations for online prediction tasks |
| title_sort | neural controlled differential equations for online prediction tasks |
| work_keys_str_mv | AT morrillj neuralcontrolleddifferentialequationsforonlinepredictiontasks AT kidgerp neuralcontrolleddifferentialequationsforonlinepredictiontasks AT yangl neuralcontrolleddifferentialequationsforonlinepredictiontasks AT lyonst neuralcontrolleddifferentialequationsforonlinepredictiontasks |