SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation
Abstract The data‐driven approaches for medium‐range weather forecasting are recently shown to be extraordinarily promising for ensemble forecasting due to their fast inference speed compared to the traditional numerical weather prediction models. However, their forecast accuracy can hardly match th...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Geophysical Union (AGU)
2023-02-01
|
| Series: | Journal of Advances in Modeling Earth Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1029/2022MS003211 |
| _version_ | 1827795937820409856 |
|---|---|
| author | Yuan Hu Lei Chen Zhibin Wang Hao Li |
| author_facet | Yuan Hu Lei Chen Zhibin Wang Hao Li |
| author_sort | Yuan Hu |
| collection | DOAJ |
| description | Abstract The data‐driven approaches for medium‐range weather forecasting are recently shown to be extraordinarily promising for ensemble forecasting due to their fast inference speed compared to the traditional numerical weather prediction models. However, their forecast accuracy can hardly match the state‐of‐the‐art operational ECMWF Integrated Forecasting System (IFS) model. Previous data‐driven approaches perform ensemble forecasting using some simple perturbation methods, like the initial condition perturbation and the Monte Carlo dropout. However, their ensemble performance is often limited arguably by the sub‐optimal ways of applying perturbation. We propose a Swin Transformer‐based Variational Recurrent Neural Network (SwinVRNN), which is a stochastic weather forecasting model combining a SwinRNN predictor with a perturbation module. SwinRNN is designed as a Swin Transformer‐based recurrent neural network, which predicts the future states deterministically. Furthermore, to model the stochasticity in the prediction, we design a perturbation module following the Variational Auto‐Encoder paradigm to learn the multivariate Gaussian distributions of a time‐variant stochastic latent variable from the data. Ensemble forecasting can be easily performed by perturbing the model features leveraging the noise sampled from the learned distribution. We also compare four categories of perturbation methods for ensemble forecasting, that is, fixed distribution perturbation, learned distribution perturbation, MC dropout, and multi model ensemble. Comparisons on the WeatherBench data set show that the learned distribution perturbation method using our SwinVRNN model achieves remarkably improved forecasting accuracy and reasonable ensemble spread due to the joint optimization of the two targets. More notably, SwinVRNN surpasses operational IFS on the surface variables of the 2‐m temperature and the 6‐hourly total precipitation at all lead times up to 5 days (Code is available at https://github.com/tpys/wwprediction). |
| first_indexed | 2024-03-11T18:58:24Z |
| format | Article |
| id | doaj.art-622301c50d9949089056ec60cc72a4f4 |
| institution | Directory Open Access Journal |
| issn | 1942-2466 |
| language | English |
| last_indexed | 2024-03-11T18:58:24Z |
| publishDate | 2023-02-01 |
| publisher | American Geophysical Union (AGU) |
| record_format | Article |
| series | Journal of Advances in Modeling Earth Systems |
| spelling | doaj.art-622301c50d9949089056ec60cc72a4f42023-10-10T14:00:43ZengAmerican Geophysical Union (AGU)Journal of Advances in Modeling Earth Systems1942-24662023-02-01152n/an/a10.1029/2022MS003211SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution PerturbationYuan Hu0Lei Chen1Zhibin Wang2Hao Li3DAMO Academy Alibaba Group Beijing ChinaDAMO Academy Alibaba Group Beijing ChinaDAMO Academy Alibaba Group Beijing ChinaArtificial Intelligence Innovation and Incubation Institute Fudan University Shanghai ChinaAbstract The data‐driven approaches for medium‐range weather forecasting are recently shown to be extraordinarily promising for ensemble forecasting due to their fast inference speed compared to the traditional numerical weather prediction models. However, their forecast accuracy can hardly match the state‐of‐the‐art operational ECMWF Integrated Forecasting System (IFS) model. Previous data‐driven approaches perform ensemble forecasting using some simple perturbation methods, like the initial condition perturbation and the Monte Carlo dropout. However, their ensemble performance is often limited arguably by the sub‐optimal ways of applying perturbation. We propose a Swin Transformer‐based Variational Recurrent Neural Network (SwinVRNN), which is a stochastic weather forecasting model combining a SwinRNN predictor with a perturbation module. SwinRNN is designed as a Swin Transformer‐based recurrent neural network, which predicts the future states deterministically. Furthermore, to model the stochasticity in the prediction, we design a perturbation module following the Variational Auto‐Encoder paradigm to learn the multivariate Gaussian distributions of a time‐variant stochastic latent variable from the data. Ensemble forecasting can be easily performed by perturbing the model features leveraging the noise sampled from the learned distribution. We also compare four categories of perturbation methods for ensemble forecasting, that is, fixed distribution perturbation, learned distribution perturbation, MC dropout, and multi model ensemble. Comparisons on the WeatherBench data set show that the learned distribution perturbation method using our SwinVRNN model achieves remarkably improved forecasting accuracy and reasonable ensemble spread due to the joint optimization of the two targets. More notably, SwinVRNN surpasses operational IFS on the surface variables of the 2‐m temperature and the 6‐hourly total precipitation at all lead times up to 5 days (Code is available at https://github.com/tpys/wwprediction).https://doi.org/10.1029/2022MS003211medium‐range weather forecastingdata‐driven methodensemble forecastlearned distribution perturbation |
| spellingShingle | Yuan Hu Lei Chen Zhibin Wang Hao Li SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation Journal of Advances in Modeling Earth Systems medium‐range weather forecasting data‐driven method ensemble forecast learned distribution perturbation |
| title | SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation |
| title_full | SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation |
| title_fullStr | SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation |
| title_full_unstemmed | SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation |
| title_short | SwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation |
| title_sort | swinvrnn a data driven ensemble forecasting model via learned distribution perturbation |
| topic | medium‐range weather forecasting data‐driven method ensemble forecast learned distribution perturbation |
| url | https://doi.org/10.1029/2022MS003211 |
| work_keys_str_mv | AT yuanhu swinvrnnadatadrivenensembleforecastingmodelvialearneddistributionperturbation AT leichen swinvrnnadatadrivenensembleforecastingmodelvialearneddistributionperturbation AT zhibinwang swinvrnnadatadrivenensembleforecastingmodelvialearneddistributionperturbation AT haoli swinvrnnadatadrivenensembleforecastingmodelvialearneddistributionperturbation |