On the choice of interpolation scheme for neural CDEs
Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations r...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Journal of Machine Learning Research
2022
|
| _version_ | 1797107779238887424 |
|---|---|
| author | Morrill, J Kidger, P Yang, L Lyons, T |
| author_facet | Morrill, J Kidger, P Yang, L Lyons, T |
| author_sort | Morrill, J |
| collection | OXFORD |
| description | Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in online prediction tasks, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that control paths for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks. |
| first_indexed | 2024-03-07T07:20:39Z |
| format | Journal article |
| id | oxford-uuid:f7b00cea-1005-4537-bddd-55203a3b60b1 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:20:39Z |
| publishDate | 2022 |
| publisher | Journal of Machine Learning Research |
| record_format | dspace |
| spelling | oxford-uuid:f7b00cea-1005-4537-bddd-55203a3b60b12022-09-27T17:31:35ZOn the choice of interpolation scheme for neural CDEsJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:f7b00cea-1005-4537-bddd-55203a3b60b1EnglishSymplectic ElementsJournal of Machine Learning Research2022Morrill, JKidger, PYang, LLyons, TNeural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in online prediction tasks, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that control paths for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks. |
| spellingShingle | Morrill, J Kidger, P Yang, L Lyons, T On the choice of interpolation scheme for neural CDEs |
| title | On the choice of interpolation scheme for neural CDEs |
| title_full | On the choice of interpolation scheme for neural CDEs |
| title_fullStr | On the choice of interpolation scheme for neural CDEs |
| title_full_unstemmed | On the choice of interpolation scheme for neural CDEs |
| title_short | On the choice of interpolation scheme for neural CDEs |
| title_sort | on the choice of interpolation scheme for neural cdes |
| work_keys_str_mv | AT morrillj onthechoiceofinterpolationschemeforneuralcdes AT kidgerp onthechoiceofinterpolationschemeforneuralcdes AT yangl onthechoiceofinterpolationschemeforneuralcdes AT lyonst onthechoiceofinterpolationschemeforneuralcdes |