Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation
Abstract Data assimilation (DA) is integrated with machine learning in order to perform entirely data‐driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as pretrained surrogate models to replace key components of the DA cycle in numerical weather predic...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Geophysical Union (AGU)
2022-03-01
|
| Series: | Journal of Advances in Modeling Earth Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1029/2021MS002843 |
| _version_ | 1797838443078746112 |
|---|---|
| author | S. G. Penny T. A. Smith T.‐C. Chen J. A. Platt H.‐Y. Lin M. Goodliff H. D. I. Abarbanel |
| author_facet | S. G. Penny T. A. Smith T.‐C. Chen J. A. Platt H.‐Y. Lin M. Goodliff H. D. I. Abarbanel |
| author_sort | S. G. Penny |
| collection | DOAJ |
| description | Abstract Data assimilation (DA) is integrated with machine learning in order to perform entirely data‐driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as pretrained surrogate models to replace key components of the DA cycle in numerical weather prediction (NWP), including the conventional numerical forecast model, the forecast error covariance matrix, and the tangent linear and adjoint models. It is shown how these RNNs can be initialized using DA methods to directly update the hidden/reservoir state with observations of the target system. The results indicate that these techniques can be applied to estimate the state of a system for the repeated initialization of short‐term forecasts, even in the absence of a traditional numerical forecast model. Further, it is demonstrated how these integrated RNN‐DA methods can scale to higher dimensions by applying domain localization and parallelization, providing a path for practical applications in NWP. |
| first_indexed | 2024-04-09T15:42:02Z |
| format | Article |
| id | doaj.art-6e9a38742bdd4c79bfb5c0158edeb25f |
| institution | Directory Open Access Journal |
| issn | 1942-2466 |
| language | English |
| last_indexed | 2024-04-09T15:42:02Z |
| publishDate | 2022-03-01 |
| publisher | American Geophysical Union (AGU) |
| record_format | Article |
| series | Journal of Advances in Modeling Earth Systems |
| spelling | doaj.art-6e9a38742bdd4c79bfb5c0158edeb25f2023-04-27T07:53:09ZengAmerican Geophysical Union (AGU)Journal of Advances in Modeling Earth Systems1942-24662022-03-01143n/an/a10.1029/2021MS002843Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State EstimationS. G. Penny0T. A. Smith1T.‐C. Chen2J. A. Platt3H.‐Y. Lin4M. Goodliff5H. D. I. Abarbanel6Cooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USACooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USACooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USADepartment of Physics University of California San Diego (UCSD) La Jolla CA USACooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USACooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USADepartment of Physics University of California San Diego (UCSD) La Jolla CA USAAbstract Data assimilation (DA) is integrated with machine learning in order to perform entirely data‐driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as pretrained surrogate models to replace key components of the DA cycle in numerical weather prediction (NWP), including the conventional numerical forecast model, the forecast error covariance matrix, and the tangent linear and adjoint models. It is shown how these RNNs can be initialized using DA methods to directly update the hidden/reservoir state with observations of the target system. The results indicate that these techniques can be applied to estimate the state of a system for the repeated initialization of short‐term forecasts, even in the absence of a traditional numerical forecast model. Further, it is demonstrated how these integrated RNN‐DA methods can scale to higher dimensions by applying domain localization and parallelization, providing a path for practical applications in NWP.https://doi.org/10.1029/2021MS002843data assimilationrecurrent neural networksmachine learningartificial intelligenceensemble kalman filter4D‐var |
| spellingShingle | S. G. Penny T. A. Smith T.‐C. Chen J. A. Platt H.‐Y. Lin M. Goodliff H. D. I. Abarbanel Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation Journal of Advances in Modeling Earth Systems data assimilation recurrent neural networks machine learning artificial intelligence ensemble kalman filter 4D‐var |
| title | Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation |
| title_full | Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation |
| title_fullStr | Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation |
| title_full_unstemmed | Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation |
| title_short | Integrating Recurrent Neural Networks With Data Assimilation for Scalable Data‐Driven State Estimation |
| title_sort | integrating recurrent neural networks with data assimilation for scalable data driven state estimation |
| topic | data assimilation recurrent neural networks machine learning artificial intelligence ensemble kalman filter 4D‐var |
| url | https://doi.org/10.1029/2021MS002843 |
| work_keys_str_mv | AT sgpenny integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation AT tasmith integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation AT tcchen integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation AT japlatt integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation AT hylin integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation AT mgoodliff integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation AT hdiabarbanel integratingrecurrentneuralnetworkswithdataassimilationforscalabledatadrivenstateestimation |