Stiff neural ordinary differential equations
Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
AIP Publishing
2021
|
| Online Access: | https://hdl.handle.net/1721.1/138719 |
| _version_ | 1811073377240088576 |
|---|---|
| author | Kim, Suyong Ji, Weiqi Deng, Sili Ma, Yingbo Rackauckas, Christopher |
| author2 | Massachusetts Institute of Technology. Department of Mechanical Engineering |
| author_facet | Massachusetts Institute of Technology. Department of Mechanical Engineering Kim, Suyong Ji, Weiqi Deng, Sili Ma, Yingbo Rackauckas, Christopher |
| author_sort | Kim, Suyong |
| collection | MIT |
| description | Neural Ordinary Differential Equations (ODE) are a promising approach to
learn dynamic models from time-series data in science and engineering
applications. This work aims at learning Neural ODE for stiff systems, which
are usually raised from chemical kinetic modeling in chemical and biological
systems. We first show the challenges of learning neural ODE in the classical
stiff ODE systems of Robertson's problem and propose techniques to mitigate the
challenges associated with scale separations in stiff systems. We then present
successful demonstrations in stiff systems of Robertson's problem and an air
pollution problem. The demonstrations show that the usage of deep networks with
rectified activations, proper scaling of the network outputs as well as loss
functions, and stabilized gradient calculations are the key techniques enabling
the learning of stiff neural ODE. The success of learning stiff neural ODE
opens up possibilities of using neural ODEs in applications with widely varying
time-scales, like chemical dynamics in energy conversion, environmental
engineering, and the life sciences. |
| first_indexed | 2024-09-23T09:32:07Z |
| format | Article |
| id | mit-1721.1/138719 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T09:32:07Z |
| publishDate | 2021 |
| publisher | AIP Publishing |
| record_format | dspace |
| spelling | mit-1721.1/1387192023-02-10T18:33:12Z Stiff neural ordinary differential equations Kim, Suyong Ji, Weiqi Deng, Sili Ma, Yingbo Rackauckas, Christopher Massachusetts Institute of Technology. Department of Mechanical Engineering Massachusetts Institute of Technology. Department of Mathematics Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODE. The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering, and the life sciences. 2021-12-17T19:15:18Z 2021-12-17T19:15:18Z 2021-09 2021-12-17T18:57:13Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/138719 Kim, Suyong, Ji, Weiqi, Deng, Sili, Ma, Yingbo and Rackauckas, Christopher. 2021. "Stiff neural ordinary differential equations." Chaos: An Interdisciplinary Journal of Nonlinear Science, 31 (9). en 10.1063/5.0060697 Chaos: An Interdisciplinary Journal of Nonlinear Science Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf AIP Publishing arXiv |
| spellingShingle | Kim, Suyong Ji, Weiqi Deng, Sili Ma, Yingbo Rackauckas, Christopher Stiff neural ordinary differential equations |
| title | Stiff neural ordinary differential equations |
| title_full | Stiff neural ordinary differential equations |
| title_fullStr | Stiff neural ordinary differential equations |
| title_full_unstemmed | Stiff neural ordinary differential equations |
| title_short | Stiff neural ordinary differential equations |
| title_sort | stiff neural ordinary differential equations |
| url | https://hdl.handle.net/1721.1/138719 |
| work_keys_str_mv | AT kimsuyong stiffneuralordinarydifferentialequations AT jiweiqi stiffneuralordinarydifferentialequations AT dengsili stiffneuralordinarydifferentialequations AT mayingbo stiffneuralordinarydifferentialequations AT rackauckaschristopher stiffneuralordinarydifferentialequations |