When best is the enemy of good – critical evaluation of performance criteria in hydrological models
<p>Performance criteria play a key role in the calibration and evaluation of hydrological models and have been extensively developed and studied, but some of the most used criteria still have unknown pitfalls. This study set out to examine counterbalancing errors, which are inherent to the Kli...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2023-07-01
|
| Series: | Hydrology and Earth System Sciences |
| Online Access: | https://hess.copernicus.org/articles/27/2397/2023/hess-27-2397-2023.pdf |
| _version_ | 1797789145780715520 |
|---|---|
| author | G. Cinkus N. Mazzilli H. Jourde A. Wunsch T. Liesch N. Ravbar Z. Chen N. Goldscheider |
| author_facet | G. Cinkus N. Mazzilli H. Jourde A. Wunsch T. Liesch N. Ravbar Z. Chen N. Goldscheider |
| author_sort | G. Cinkus |
| collection | DOAJ |
| description | <p>Performance criteria play a key role in the calibration and evaluation of hydrological models and have been extensively developed and studied, but some of the most used criteria still have unknown pitfalls. This study set out to examine counterbalancing errors, which are inherent to the Kling–Gupta efficiency (KGE) and its variants. A total of nine performance criteria – including the KGE and its variants, as well as the
Nash–Sutcliffe efficiency (NSE) and the modified index of agreement
(<span class="inline-formula"><i>d</i><sub>1</sub></span>) – were analysed using synthetic time series and a real case
study. Results showed that, when assessing a simulation, the score of the KGE and
some of its variants can be increased by concurrent overestimation and underestimation of discharge. These counterbalancing errors may favour bias
and variability parameters, therefore preserving an overall high score of
the performance criteria. As bias and variability parameters generally
account for two-thirds of the weight in the equation of performance criteria such as the KGE, this can lead to an overall higher criterion score without being associated with an increase in model relevance. We recommend using (i) performance criteria that are not or less prone to counterbalancing errors (<span class="inline-formula"><i>d</i><sub>1</sub></span>, modified KGE, non-parametric KGE, diagnostic efficiency) and/or (ii) scaling factors in the equation to reduce the influence of relative parameters.</p> |
| first_indexed | 2024-03-13T01:46:35Z |
| format | Article |
| id | doaj.art-676d9bc11fe146b19c1b92d228705eeb |
| institution | Directory Open Access Journal |
| issn | 1027-5606 1607-7938 |
| language | English |
| last_indexed | 2024-03-13T01:46:35Z |
| publishDate | 2023-07-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Hydrology and Earth System Sciences |
| spelling | doaj.art-676d9bc11fe146b19c1b92d228705eeb2023-07-03T07:58:18ZengCopernicus PublicationsHydrology and Earth System Sciences1027-56061607-79382023-07-01272397241110.5194/hess-27-2397-2023When best is the enemy of good – critical evaluation of performance criteria in hydrological modelsG. Cinkus0N. Mazzilli1H. Jourde2A. Wunsch3T. Liesch4N. Ravbar5Z. Chen6N. Goldscheider7HydroSciences Montpellier (HSM), CNRS, IRD, Univ. Montpellier, 34090 Montpellier, FranceUMR 1114 EMMAH (AU-INRAE), Université d'Avignon, 84000 Avignon, FranceHydroSciences Montpellier (HSM), CNRS, IRD, Univ. Montpellier, 34090 Montpellier, FranceInstitute of Applied Geosciences, Karlsruhe Institute of Technology (KIT), Kaiserstr. 12, 76131 Karlsruhe, GermanyInstitute of Applied Geosciences, Karlsruhe Institute of Technology (KIT), Kaiserstr. 12, 76131 Karlsruhe, GermanyKarst Research Institute, ZRC SAZU, Titov trg 2, 6230 Postojna, SloveniaInstitute of Groundwater Management, Technical University of Dresden, 01062 Dresden, GermanyInstitute of Applied Geosciences, Karlsruhe Institute of Technology (KIT), Kaiserstr. 12, 76131 Karlsruhe, Germany<p>Performance criteria play a key role in the calibration and evaluation of hydrological models and have been extensively developed and studied, but some of the most used criteria still have unknown pitfalls. This study set out to examine counterbalancing errors, which are inherent to the Kling–Gupta efficiency (KGE) and its variants. A total of nine performance criteria – including the KGE and its variants, as well as the Nash–Sutcliffe efficiency (NSE) and the modified index of agreement (<span class="inline-formula"><i>d</i><sub>1</sub></span>) – were analysed using synthetic time series and a real case study. Results showed that, when assessing a simulation, the score of the KGE and some of its variants can be increased by concurrent overestimation and underestimation of discharge. These counterbalancing errors may favour bias and variability parameters, therefore preserving an overall high score of the performance criteria. As bias and variability parameters generally account for two-thirds of the weight in the equation of performance criteria such as the KGE, this can lead to an overall higher criterion score without being associated with an increase in model relevance. We recommend using (i) performance criteria that are not or less prone to counterbalancing errors (<span class="inline-formula"><i>d</i><sub>1</sub></span>, modified KGE, non-parametric KGE, diagnostic efficiency) and/or (ii) scaling factors in the equation to reduce the influence of relative parameters.</p>https://hess.copernicus.org/articles/27/2397/2023/hess-27-2397-2023.pdf |
| spellingShingle | G. Cinkus N. Mazzilli H. Jourde A. Wunsch T. Liesch N. Ravbar Z. Chen N. Goldscheider When best is the enemy of good – critical evaluation of performance criteria in hydrological models Hydrology and Earth System Sciences |
| title | When best is the enemy of good – critical evaluation of performance criteria in hydrological models |
| title_full | When best is the enemy of good – critical evaluation of performance criteria in hydrological models |
| title_fullStr | When best is the enemy of good – critical evaluation of performance criteria in hydrological models |
| title_full_unstemmed | When best is the enemy of good – critical evaluation of performance criteria in hydrological models |
| title_short | When best is the enemy of good – critical evaluation of performance criteria in hydrological models |
| title_sort | when best is the enemy of good critical evaluation of performance criteria in hydrological models |
| url | https://hess.copernicus.org/articles/27/2397/2023/hess-27-2397-2023.pdf |
| work_keys_str_mv | AT gcinkus whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT nmazzilli whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT hjourde whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT awunsch whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT tliesch whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT nravbar whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT zchen whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels AT ngoldscheider whenbestistheenemyofgoodcriticalevaluationofperformancecriteriainhydrologicalmodels |