Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence
Abstract The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to th...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Geophysical Union (AGU)
2023-12-01
|
| Series: | Journal of Advances in Modeling Earth Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1029/2023MS003792 |
| _version_ | 1797356830227169280 |
|---|---|
| author | Timothy A. Smith Stephen G. Penny Jason A. Platt Tse‐Chun Chen |
| author_facet | Timothy A. Smith Stephen G. Penny Jason A. Platt Tse‐Chun Chen |
| author_sort | Timothy A. Smith |
| collection | DOAJ |
| description | Abstract The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi‐Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures. |
| first_indexed | 2024-03-08T14:35:31Z |
| format | Article |
| id | doaj.art-b2fb9489890245869709bb8a9be9e472 |
| institution | Directory Open Access Journal |
| issn | 1942-2466 |
| language | English |
| last_indexed | 2024-03-08T14:35:31Z |
| publishDate | 2023-12-01 |
| publisher | American Geophysical Union (AGU) |
| record_format | Article |
| series | Journal of Advances in Modeling Earth Systems |
| spelling | doaj.art-b2fb9489890245869709bb8a9be9e4722024-01-12T05:31:23ZengAmerican Geophysical Union (AGU)Journal of Advances in Modeling Earth Systems1942-24662023-12-011512n/an/a10.1029/2023MS003792Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical TurbulenceTimothy A. Smith0Stephen G. Penny1Jason A. Platt2Tse‐Chun Chen3Cooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USACooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado Boulder Boulder CO USAUniversity of California San Diego (UCSD) La Jolla CA USAPacific Northwest National Laboratory Richland WA USAAbstract The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi‐Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.https://doi.org/10.1029/2023MS003792machine learningrecurrent neural networksnumerical weather predictiongeophysical fluid dynamics |
| spellingShingle | Timothy A. Smith Stephen G. Penny Jason A. Platt Tse‐Chun Chen Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence Journal of Advances in Modeling Earth Systems machine learning recurrent neural networks numerical weather prediction geophysical fluid dynamics |
| title | Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence |
| title_full | Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence |
| title_fullStr | Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence |
| title_full_unstemmed | Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence |
| title_short | Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence |
| title_sort | temporal subsampling diminishes small spatial scales in recurrent neural network emulators of geophysical turbulence |
| topic | machine learning recurrent neural networks numerical weather prediction geophysical fluid dynamics |
| url | https://doi.org/10.1029/2023MS003792 |
| work_keys_str_mv | AT timothyasmith temporalsubsamplingdiminishessmallspatialscalesinrecurrentneuralnetworkemulatorsofgeophysicalturbulence AT stephengpenny temporalsubsamplingdiminishessmallspatialscalesinrecurrentneuralnetworkemulatorsofgeophysicalturbulence AT jasonaplatt temporalsubsamplingdiminishessmallspatialscalesinrecurrentneuralnetworkemulatorsofgeophysicalturbulence AT tsechunchen temporalsubsamplingdiminishessmallspatialscalesinrecurrentneuralnetworkemulatorsofgeophysicalturbulence |