A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context
<p>An increasing number of flood forecasting services assess and communicate the uncertainty associated with their forecasts. While obtaining reliable forecasts is a key issue, it is a challenging task, especially when forecasting high flows in an extrapolation context, i.e. when the event mag...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2020-04-01
|
| Series: | Hydrology and Earth System Sciences |
| Online Access: | https://www.hydrol-earth-syst-sci.net/24/2017/2020/hess-24-2017-2020.pdf |
| _version_ | 1819266327371579392 |
|---|---|
| author | L. Berthet F. Bourgin F. Bourgin C. Perrin J. Viatgé R. Marty O. Piotte |
| author_facet | L. Berthet F. Bourgin F. Bourgin C. Perrin J. Viatgé R. Marty O. Piotte |
| author_sort | L. Berthet |
| collection | DOAJ |
| description | <p>An increasing number of flood forecasting services assess and communicate the uncertainty associated with their forecasts. While obtaining reliable forecasts is a key issue, it is a challenging task, especially when forecasting high flows in an extrapolation context, i.e. when the event magnitude is larger than what was observed before. In this study, we present a crash-testing framework that evaluates the quality of hydrological forecasts in an extrapolation context. The experiment set-up is based on (i) a large set of catchments in France, (ii) the GRP rainfall–runoff model designed for flood forecasting and used by the French operational services and (iii) an empirical hydrologic uncertainty processor designed to estimate conditional predictive uncertainty from the hydrological model residuals. The variants of the uncertainty processor used in this study differ in the data transformation they use (log, Box–Cox and log–sinh) to account for heteroscedasticity and the evolution of the other properties of the predictive distribution with the discharge magnitude. Different data subsets were selected based on a preliminary event selection. Various aspects of the probabilistic performance of the variants of the hydrologic uncertainty processor, reliability, sharpness and overall quality were evaluated. Overall, the results highlight the challenge of uncertainty quantification when forecasting high flows. They show a significant drop in reliability when forecasting high flows in an extrapolation context and considerable variability among catchments and across lead times. The increase in statistical treatment complexity did not result in significant improvement, which suggests that a parsimonious and easily understandable data transformation such as the log transformation or the Box–Cox transformation can be a reasonable choice for flood forecasting.</p> |
| first_indexed | 2024-12-23T20:59:31Z |
| format | Article |
| id | doaj.art-0fd8b9921025459484e194a0e99bb7b3 |
| institution | Directory Open Access Journal |
| issn | 1027-5606 1607-7938 |
| language | English |
| last_indexed | 2024-12-23T20:59:31Z |
| publishDate | 2020-04-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Hydrology and Earth System Sciences |
| spelling | doaj.art-0fd8b9921025459484e194a0e99bb7b32022-12-21T17:31:24ZengCopernicus PublicationsHydrology and Earth System Sciences1027-56061607-79382020-04-01242017204110.5194/hess-24-2017-2020A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation contextL. Berthet0F. Bourgin1F. Bourgin2C. Perrin3J. Viatgé4R. Marty5O. Piotte6DREAL Centre-Val de Loire, Loire Cher & Indre Flood Forecasting Service, Orléans, FranceGERS-LEE, Univ Gustave Eiffel, IFSTTAR, 44344 Bouguenais, FranceUniversité Paris-Saclay, INRAE, UR HYCAR, 92160 Antony, FranceUniversité Paris-Saclay, INRAE, UR HYCAR, 92160 Antony, FranceUniversité Paris-Saclay, INRAE, UR HYCAR, 92160 Antony, FranceDREAL Centre-Val de Loire, Loire Cher & Indre Flood Forecasting Service, Orléans, FranceMinistry for the Ecological and Inclusive Transition, SCHAPI, Toulouse, France<p>An increasing number of flood forecasting services assess and communicate the uncertainty associated with their forecasts. While obtaining reliable forecasts is a key issue, it is a challenging task, especially when forecasting high flows in an extrapolation context, i.e. when the event magnitude is larger than what was observed before. In this study, we present a crash-testing framework that evaluates the quality of hydrological forecasts in an extrapolation context. The experiment set-up is based on (i) a large set of catchments in France, (ii) the GRP rainfall–runoff model designed for flood forecasting and used by the French operational services and (iii) an empirical hydrologic uncertainty processor designed to estimate conditional predictive uncertainty from the hydrological model residuals. The variants of the uncertainty processor used in this study differ in the data transformation they use (log, Box–Cox and log–sinh) to account for heteroscedasticity and the evolution of the other properties of the predictive distribution with the discharge magnitude. Different data subsets were selected based on a preliminary event selection. Various aspects of the probabilistic performance of the variants of the hydrologic uncertainty processor, reliability, sharpness and overall quality were evaluated. Overall, the results highlight the challenge of uncertainty quantification when forecasting high flows. They show a significant drop in reliability when forecasting high flows in an extrapolation context and considerable variability among catchments and across lead times. The increase in statistical treatment complexity did not result in significant improvement, which suggests that a parsimonious and easily understandable data transformation such as the log transformation or the Box–Cox transformation can be a reasonable choice for flood forecasting.</p>https://www.hydrol-earth-syst-sci.net/24/2017/2020/hess-24-2017-2020.pdf |
| spellingShingle | L. Berthet F. Bourgin F. Bourgin C. Perrin J. Viatgé R. Marty O. Piotte A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context Hydrology and Earth System Sciences |
| title | A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context |
| title_full | A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context |
| title_fullStr | A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context |
| title_full_unstemmed | A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context |
| title_short | A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context |
| title_sort | crash testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context |
| url | https://www.hydrol-earth-syst-sci.net/24/2017/2020/hess-24-2017-2020.pdf |
| work_keys_str_mv | AT lberthet acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT fbourgin acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT fbourgin acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT cperrin acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT jviatge acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT rmarty acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT opiotte acrashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT lberthet crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT fbourgin crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT fbourgin crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT cperrin crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT jviatge crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT rmarty crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext AT opiotte crashtestingframeworkforpredictiveuncertaintyassessmentwhenforecastinghighflowsinanextrapolationcontext |