An Optimal Linear Transformation for Data Assimilation

Abstract Linear transformations are widely used in data assimilation for covariance modeling, for reducing dimensionality (such as averaging dense observations to form “superobs”), and for managing sampling error in ensemble data assimilation. Here we describe a linear transformation that is optimal...

Full description

Bibliographic Details
Main Authors: Chris Snyder, Gregory J. Hakim
Format: Article
Language:English
Published: American Geophysical Union (AGU) 2022-06-01
Series:Journal of Advances in Modeling Earth Systems
Subjects:
Online Access:https://doi.org/10.1029/2021MS002937
_version_ 1818194610423332864
author Chris Snyder
Gregory J. Hakim
author_facet Chris Snyder
Gregory J. Hakim
author_sort Chris Snyder
collection DOAJ
description Abstract Linear transformations are widely used in data assimilation for covariance modeling, for reducing dimensionality (such as averaging dense observations to form “superobs”), and for managing sampling error in ensemble data assimilation. Here we describe a linear transformation that is optimal in the sense that, in the transformed space, the state variables and observations have uncorrelated errors, and a diagonal gain matrix in the update step. We conjecture, and provide numerical evidence, that the transformation is the best possible to precede covariance localization in an ensemble Kalman filter. A central feature of this transformation in the update step are scalars, which we term canonical observation operators (COOs), that relate pairs of transformed observations and state variables and rank‐order those pairs by their influence in the update. We show for an idealized problem that sample‐based estimates of the COOs, in conjunction with covariance localization for the sample covariance, can approximate well the true values, but a practical implementation of the transformation for high‐dimensional applications remains a subject for future research. The COOs also completely describe important properties of the update step, such as observation‐state mutual information, signal‐to‐noise and degrees of freedom for signal, and so give new insights, including relations among reduced‐rank approximations to variational schemes, particle‐filter weight degeneracy, and the local ensemble transform Kalman filter.
first_indexed 2024-12-12T01:05:02Z
format Article
id doaj.art-0f0077c858f9455b93d13e5a0c20599e
institution Directory Open Access Journal
issn 1942-2466
language English
last_indexed 2024-12-12T01:05:02Z
publishDate 2022-06-01
publisher American Geophysical Union (AGU)
record_format Article
series Journal of Advances in Modeling Earth Systems
spelling doaj.art-0f0077c858f9455b93d13e5a0c20599e2022-12-22T00:43:36ZengAmerican Geophysical Union (AGU)Journal of Advances in Modeling Earth Systems1942-24662022-06-01146n/an/a10.1029/2021MS002937An Optimal Linear Transformation for Data AssimilationChris Snyder0Gregory J. Hakim1National Center for Atmospheric Research Boulder CO USADepartment of Atmospheric Sciences University of Washington Seattle WAAbstract Linear transformations are widely used in data assimilation for covariance modeling, for reducing dimensionality (such as averaging dense observations to form “superobs”), and for managing sampling error in ensemble data assimilation. Here we describe a linear transformation that is optimal in the sense that, in the transformed space, the state variables and observations have uncorrelated errors, and a diagonal gain matrix in the update step. We conjecture, and provide numerical evidence, that the transformation is the best possible to precede covariance localization in an ensemble Kalman filter. A central feature of this transformation in the update step are scalars, which we term canonical observation operators (COOs), that relate pairs of transformed observations and state variables and rank‐order those pairs by their influence in the update. We show for an idealized problem that sample‐based estimates of the COOs, in conjunction with covariance localization for the sample covariance, can approximate well the true values, but a practical implementation of the transformation for high‐dimensional applications remains a subject for future research. The COOs also completely describe important properties of the update step, such as observation‐state mutual information, signal‐to‐noise and degrees of freedom for signal, and so give new insights, including relations among reduced‐rank approximations to variational schemes, particle‐filter weight degeneracy, and the local ensemble transform Kalman filter.https://doi.org/10.1029/2021MS002937data assimilationkalman filtercovariance localizationlinear transformationcanonical correlation analysis
spellingShingle Chris Snyder
Gregory J. Hakim
An Optimal Linear Transformation for Data Assimilation
Journal of Advances in Modeling Earth Systems
data assimilation
kalman filter
covariance localization
linear transformation
canonical correlation analysis
title An Optimal Linear Transformation for Data Assimilation
title_full An Optimal Linear Transformation for Data Assimilation
title_fullStr An Optimal Linear Transformation for Data Assimilation
title_full_unstemmed An Optimal Linear Transformation for Data Assimilation
title_short An Optimal Linear Transformation for Data Assimilation
title_sort optimal linear transformation for data assimilation
topic data assimilation
kalman filter
covariance localization
linear transformation
canonical correlation analysis
url https://doi.org/10.1029/2021MS002937
work_keys_str_mv AT chrissnyder anoptimallineartransformationfordataassimilation
AT gregoryjhakim anoptimallineartransformationfordataassimilation
AT chrissnyder optimallineartransformationfordataassimilation
AT gregoryjhakim optimallineartransformationfordataassimilation