On the relationship between reliability diagrams and the ‘signal-to-noise paradox’
The ‘signal-to-noise paradox’ for seasonal forecasts of the winter NAO is often described as an ‘underconfident’ forecast and measured using the ratio-of-predictable components metric (RPC). However, comparison of RPC with other measures of forecast confidence, such as spread-error ratios, can give...
| Main Authors: | , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
American Geophysical Union
2023
|
| _version_ | 1797110838457270272 |
|---|---|
| author | Strommen, K MacRae, M Christensen, H |
| author_facet | Strommen, K MacRae, M Christensen, H |
| author_sort | Strommen, K |
| collection | OXFORD |
| description | The ‘signal-to-noise paradox’ for seasonal forecasts of the winter NAO is often described as an ‘underconfident’ forecast and measured using the ratio-of-predictable components metric (RPC). However, comparison of RPC with other measures of forecast confidence, such as spread-error ratios, can give conflicting impressions, challenging this informal description. We show, using a linear statistical model, that the ‘paradox’ is equivalent to a situation where the reliability diagram of any percentile forecast has a slope exceeding 1. The relationship with spread-error ratios is shown to be far less direct. We furthermore compute reliability diagrams of winter NAO forecasts using seasonal hindcasts from the European Centre for Medium-range Weather Forecasts and the UK Meteoro logical Office. While these broadly exhibit slopes exceeding 1, there is evidence of asymmetry between upper and lower terciles, indicating a potential violation of linearity/Gaussianity. The limitations and benefits of reliability diagrams as a diagnostic tool are discussed. |
| first_indexed | 2024-03-07T08:01:59Z |
| format | Journal article |
| id | oxford-uuid:5a3f44fc-8351-4695-bdff-232b5e223acd |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:01:59Z |
| publishDate | 2023 |
| publisher | American Geophysical Union |
| record_format | dspace |
| spelling | oxford-uuid:5a3f44fc-8351-4695-bdff-232b5e223acd2023-10-03T09:16:37ZOn the relationship between reliability diagrams and the ‘signal-to-noise paradox’Journal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:5a3f44fc-8351-4695-bdff-232b5e223acdEnglishSymplectic ElementsAmerican Geophysical Union2023Strommen, KMacRae, MChristensen, HThe ‘signal-to-noise paradox’ for seasonal forecasts of the winter NAO is often described as an ‘underconfident’ forecast and measured using the ratio-of-predictable components metric (RPC). However, comparison of RPC with other measures of forecast confidence, such as spread-error ratios, can give conflicting impressions, challenging this informal description. We show, using a linear statistical model, that the ‘paradox’ is equivalent to a situation where the reliability diagram of any percentile forecast has a slope exceeding 1. The relationship with spread-error ratios is shown to be far less direct. We furthermore compute reliability diagrams of winter NAO forecasts using seasonal hindcasts from the European Centre for Medium-range Weather Forecasts and the UK Meteoro logical Office. While these broadly exhibit slopes exceeding 1, there is evidence of asymmetry between upper and lower terciles, indicating a potential violation of linearity/Gaussianity. The limitations and benefits of reliability diagrams as a diagnostic tool are discussed. |
| spellingShingle | Strommen, K MacRae, M Christensen, H On the relationship between reliability diagrams and the ‘signal-to-noise paradox’ |
| title | On the relationship between reliability diagrams and the ‘signal-to-noise paradox’ |
| title_full | On the relationship between reliability diagrams and the ‘signal-to-noise paradox’ |
| title_fullStr | On the relationship between reliability diagrams and the ‘signal-to-noise paradox’ |
| title_full_unstemmed | On the relationship between reliability diagrams and the ‘signal-to-noise paradox’ |
| title_short | On the relationship between reliability diagrams and the ‘signal-to-noise paradox’ |
| title_sort | on the relationship between reliability diagrams and the signal to noise paradox |
| work_keys_str_mv | AT strommenk ontherelationshipbetweenreliabilitydiagramsandthesignaltonoiseparadox AT macraem ontherelationshipbetweenreliabilitydiagramsandthesignaltonoiseparadox AT christensenh ontherelationshipbetweenreliabilitydiagramsandthesignaltonoiseparadox |