Calibrating ensemble reliability whilst preserving spatial structure
Ensemble forecasts aim to improve decision-making by predicting a set of possible outcomes. Ideally, these would provide probabilities which are both sharp and reliable. In practice, the models, data assimilation and ensemble perturbation systems are all imperfect, leading to deficiencies in the pre...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Stockholm University Press
2014-03-01
|
| Series: | Tellus: Series A, Dynamic Meteorology and Oceanography |
| Subjects: | |
| Online Access: | http://www.tellusa.net/index.php/tellusa/article/download/22662/pdf_1 |
| _version_ | 1818040988594077696 |
|---|---|
| author | Jonathan Flowerdew |
| author_facet | Jonathan Flowerdew |
| author_sort | Jonathan Flowerdew |
| collection | DOAJ |
| description | Ensemble forecasts aim to improve decision-making by predicting a set of possible outcomes. Ideally, these would provide probabilities which are both sharp and reliable. In practice, the models, data assimilation and ensemble perturbation systems are all imperfect, leading to deficiencies in the predicted probabilities. This paper presents an ensemble post-processing scheme which directly targets local reliability, calibrating both climatology and ensemble dispersion in one coherent operation. It makes minimal assumptions about the underlying statistical distributions, aiming to extract as much information as possible from the original dynamic forecasts and support statistically awkward variables such as precipitation. The output is a set of ensemble members preserving the spatial, temporal and inter-variable structure from the raw forecasts, which should be beneficial to downstream applications such as hydrological models. The calibration is tested on three leading 15-d ensemble systems, and their aggregation into a simple multimodel ensemble. Results are presented for 12 h, 1° scale over Europe for a range of surface variables, including precipitation. The scheme is very effective at removing unreliability from the raw forecasts, whilst generally preserving or improving statistical resolution. In most cases, these benefits extend to the rarest events at each location within the 2-yr verification period. The reliability and resolution are generally equivalent or superior to those achieved using a Local Quantile-Quantile Transform, an established calibration method which generalises bias correction. The value of preserving spatial structure is demonstrated by the fact that 3×3 averages derived from grid-scale precipitation calibration perform almost as well as direct calibration at 3×3 scale, and much better than a similar test neglecting the spatial relationships. Some remaining issues are discussed regarding the finite size of the output ensemble, variables such as sea-level pressure which are very reliable to start with, and the best way to handle derived variables such as dewpoint depression. |
| first_indexed | 2024-12-10T08:23:16Z |
| format | Article |
| id | doaj.art-27e37a4922054654aa12772f0fc21a03 |
| institution | Directory Open Access Journal |
| issn | 0280-6495 1600-0870 |
| language | English |
| last_indexed | 2024-12-10T08:23:16Z |
| publishDate | 2014-03-01 |
| publisher | Stockholm University Press |
| record_format | Article |
| series | Tellus: Series A, Dynamic Meteorology and Oceanography |
| spelling | doaj.art-27e37a4922054654aa12772f0fc21a032022-12-22T01:56:18ZengStockholm University PressTellus: Series A, Dynamic Meteorology and Oceanography0280-64951600-08702014-03-0166012010.3402/tellusa.v66.2266222662Calibrating ensemble reliability whilst preserving spatial structureJonathan Flowerdew0Met Office, Exeter EX1 3PB, United KingdomEnsemble forecasts aim to improve decision-making by predicting a set of possible outcomes. Ideally, these would provide probabilities which are both sharp and reliable. In practice, the models, data assimilation and ensemble perturbation systems are all imperfect, leading to deficiencies in the predicted probabilities. This paper presents an ensemble post-processing scheme which directly targets local reliability, calibrating both climatology and ensemble dispersion in one coherent operation. It makes minimal assumptions about the underlying statistical distributions, aiming to extract as much information as possible from the original dynamic forecasts and support statistically awkward variables such as precipitation. The output is a set of ensemble members preserving the spatial, temporal and inter-variable structure from the raw forecasts, which should be beneficial to downstream applications such as hydrological models. The calibration is tested on three leading 15-d ensemble systems, and their aggregation into a simple multimodel ensemble. Results are presented for 12 h, 1° scale over Europe for a range of surface variables, including precipitation. The scheme is very effective at removing unreliability from the raw forecasts, whilst generally preserving or improving statistical resolution. In most cases, these benefits extend to the rarest events at each location within the 2-yr verification period. The reliability and resolution are generally equivalent or superior to those achieved using a Local Quantile-Quantile Transform, an established calibration method which generalises bias correction. The value of preserving spatial structure is demonstrated by the fact that 3×3 averages derived from grid-scale precipitation calibration perform almost as well as direct calibration at 3×3 scale, and much better than a similar test neglecting the spatial relationships. Some remaining issues are discussed regarding the finite size of the output ensemble, variables such as sea-level pressure which are very reliable to start with, and the best way to handle derived variables such as dewpoint depression.http://www.tellusa.net/index.php/tellusa/article/download/22662/pdf_1Brier Skill Scoreclimatological thresholdsEnsemble Copula CouplingLocal Quantile-Quantile Transformmedium rangemultimodel ensemblerank histogramTIGGE |
| spellingShingle | Jonathan Flowerdew Calibrating ensemble reliability whilst preserving spatial structure Tellus: Series A, Dynamic Meteorology and Oceanography Brier Skill Score climatological thresholds Ensemble Copula Coupling Local Quantile-Quantile Transform medium range multimodel ensemble rank histogram TIGGE |
| title | Calibrating ensemble reliability whilst preserving spatial structure |
| title_full | Calibrating ensemble reliability whilst preserving spatial structure |
| title_fullStr | Calibrating ensemble reliability whilst preserving spatial structure |
| title_full_unstemmed | Calibrating ensemble reliability whilst preserving spatial structure |
| title_short | Calibrating ensemble reliability whilst preserving spatial structure |
| title_sort | calibrating ensemble reliability whilst preserving spatial structure |
| topic | Brier Skill Score climatological thresholds Ensemble Copula Coupling Local Quantile-Quantile Transform medium range multimodel ensemble rank histogram TIGGE |
| url | http://www.tellusa.net/index.php/tellusa/article/download/22662/pdf_1 |
| work_keys_str_mv | AT jonathanflowerdew calibratingensemblereliabilitywhilstpreservingspatialstructure |