Cross-validation of bias-corrected climate simulations is misleading
<p>We demonstrate both analytically and with a modelling example that cross-validation of free-running bias-corrected climate change simulations against observations is misleading. The underlying reasoning is as follows: a cross-validation can have in principle two outcomes. A negative (in...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2018-09-01
|
| Series: | Hydrology and Earth System Sciences |
| Online Access: | https://www.hydrol-earth-syst-sci.net/22/4867/2018/hess-22-4867-2018.pdf |
| _version_ | 1818511054396719104 |
|---|---|
| author | D. Maraun M. Widmann |
| author_facet | D. Maraun M. Widmann |
| author_sort | D. Maraun |
| collection | DOAJ |
| description | <p>We demonstrate both analytically and with a modelling example that
cross-validation of free-running bias-corrected climate change simulations
against observations is misleading. The underlying reasoning is as follows: a
cross-validation can have in principle two outcomes. A negative (in the sense
of not rejecting a null hypothesis), if the residual bias in the validation
period after bias correction vanishes; and a positive, if the residual bias
in the validation period after bias correction is large. It can be shown
analytically that the residual bias depends solely on the difference between
the simulated and observed change between calibration and validation periods.
This change, however, depends mainly on the realizations of internal
variability in the observations and climate model. As a consequence, the
outcome of a cross-validation is also dominated by internal variability, and
does not allow for any conclusion about the sensibility of a bias correction.
In particular, a sensible bias correction may be rejected (false positive)
and a non-sensible bias correction may be accepted (false negative). We
therefore propose to avoid cross-validation when evaluating bias correction
of free-running bias-corrected climate change simulations against
observations. Instead, one should evaluate non-calibrated temporal, spatial
and process-based aspects.</p> |
| first_indexed | 2024-12-10T23:28:18Z |
| format | Article |
| id | doaj.art-b77f3206f1ec4ed7853bf392dcb1bcd2 |
| institution | Directory Open Access Journal |
| issn | 1027-5606 1607-7938 |
| language | English |
| last_indexed | 2024-12-10T23:28:18Z |
| publishDate | 2018-09-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Hydrology and Earth System Sciences |
| spelling | doaj.art-b77f3206f1ec4ed7853bf392dcb1bcd22022-12-22T01:29:29ZengCopernicus PublicationsHydrology and Earth System Sciences1027-56061607-79382018-09-01224867487310.5194/hess-22-4867-2018Cross-validation of bias-corrected climate simulations is misleadingD. Maraun0M. Widmann1Wegener Center for Climate and Global Change, University of Graz, Brandhofgasse 5, 8010 Graz, AustriaSchool of Geography, Earth and Environmental Sciences, University of Birmingham, Birmingham, B15 2TT, UK<p>We demonstrate both analytically and with a modelling example that cross-validation of free-running bias-corrected climate change simulations against observations is misleading. The underlying reasoning is as follows: a cross-validation can have in principle two outcomes. A negative (in the sense of not rejecting a null hypothesis), if the residual bias in the validation period after bias correction vanishes; and a positive, if the residual bias in the validation period after bias correction is large. It can be shown analytically that the residual bias depends solely on the difference between the simulated and observed change between calibration and validation periods. This change, however, depends mainly on the realizations of internal variability in the observations and climate model. As a consequence, the outcome of a cross-validation is also dominated by internal variability, and does not allow for any conclusion about the sensibility of a bias correction. In particular, a sensible bias correction may be rejected (false positive) and a non-sensible bias correction may be accepted (false negative). We therefore propose to avoid cross-validation when evaluating bias correction of free-running bias-corrected climate change simulations against observations. Instead, one should evaluate non-calibrated temporal, spatial and process-based aspects.</p>https://www.hydrol-earth-syst-sci.net/22/4867/2018/hess-22-4867-2018.pdf |
| spellingShingle | D. Maraun M. Widmann Cross-validation of bias-corrected climate simulations is misleading Hydrology and Earth System Sciences |
| title | Cross-validation of bias-corrected climate simulations is misleading |
| title_full | Cross-validation of bias-corrected climate simulations is misleading |
| title_fullStr | Cross-validation of bias-corrected climate simulations is misleading |
| title_full_unstemmed | Cross-validation of bias-corrected climate simulations is misleading |
| title_short | Cross-validation of bias-corrected climate simulations is misleading |
| title_sort | cross validation of bias corrected climate simulations is misleading |
| url | https://www.hydrol-earth-syst-sci.net/22/4867/2018/hess-22-4867-2018.pdf |
| work_keys_str_mv | AT dmaraun crossvalidationofbiascorrectedclimatesimulationsismisleading AT mwidmann crossvalidationofbiascorrectedclimatesimulationsismisleading |