Preprints
https://doi.org/10.5194/hess-2022-380
https://doi.org/10.5194/hess-2022-380
15 Nov 2022
 | 15 Nov 2022
Status: this preprint is currently under review for the journal HESS.

When best is the enemy of good – critical evaluation of performance criteria in hydrological models

Guillaume Cinkus, Naomi Mazzilli, Hervé Jourde, Andreas Wunsch, Tanja Liesch, Nataša Ravbar, Zhao Chen, and Nico Goldscheider

Abstract. Performance criteria play a key role in the calibration and evaluation of hydrological models and have been extensively developed and studied, but some of the most used criteria still have unknown pitfalls. This study set out to examine counterbalancing errors, which are inherent to the Kling-Gupta Efficiency (KGE) and its variants. A total of nine performance criteria – including the KGE and its variants, as well as the Nash-Sutcliffe Efficiency (NSE) and the refined version of the Willmott’s index of agreement (dr) – were analysed using synthetic time series and a real case study. Results showed that, assessing a simulation, the score of the KGE and some of its variants can be increased by concurrent over- and underestimation of discharge. These counterbalancing errors may favour bias and variability parameters, therefore preserving an overall high score of the performance criteria. As bias and variability parameters generally account for 2/3 of the weight in the equation of performance criteria such as the KGE, this can lead to an overall higher criterion score without being associated to an increase in model relevance. We recommend using (i) performance criteria that are not or less prone to counterbalancing errors (NSE, dr, modified KGE, non-parametric KGE, Diagnostic Efficiency) in a multi-criteria framework, and/or (ii) scaling factors in the equation to reduce the influence of relative parameters.

Guillaume Cinkus et al.

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CC1: 'Comment on hess-2022-380', Charles Onyutha, 12 Dec 2022
    • AC1: 'Reply on CC1', Guillaume Cinkus, 29 Jan 2023
  • CC2: 'Comment on hess-2022-380', John Ding, 22 Dec 2022
    • AC2: 'Reply on CC2', Guillaume Cinkus, 29 Jan 2023
  • RC1: 'Comment on hess-2022-380', Anonymous Referee #1, 07 Mar 2023
  • RC2: 'Comment on hess-2022-380', Anonymous Referee #2, 21 Mar 2023

Guillaume Cinkus et al.

Guillaume Cinkus et al.

Viewed

Total article views: 861 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
631 214 16 861 4 6
  • HTML: 631
  • PDF: 214
  • XML: 16
  • Total: 861
  • BibTeX: 4
  • EndNote: 6
Views and downloads (calculated since 15 Nov 2022)
Cumulative views and downloads (calculated since 15 Nov 2022)

Viewed (geographical distribution)

Total article views: 825 (including HTML, PDF, and XML) Thereof 825 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 22 Mar 2023
Download
Short summary
The Kling-Gupta Efficiency (KGE) is a performance criterion extensively used to evaluate hydrological models. We conduct a critical study on the KGE and its variant to examine counterbalancing errors. Results show that, assessing a simulation, concurrent over- and underestimation of discharge can lead to an overall higher criterion score without being associated to an increase in model relevance. We suggest to carefully choose performance criteria and to use scaling factors.