Preprints
https://doi.org/10.5194/hess-2024-59
https://doi.org/10.5194/hess-2024-59
29 Feb 2024
 | 29 Feb 2024
Status: this preprint is currently under review for the journal HESS.

Technical Note: The Divide and Measure Nonconformity

Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, and Jakob Zscheischler

Abstract. The evaluation of model performance is an essential part of hydrological modeling. However, leveraging the full information that performance criteria provide, requires a deep understanding of their properties. This Technical Note focuses on a rather counterintuitive aspect of the perhaps most widely used hydrological metric, the Nash-Sutcliffe Efficiency (NSE). Specifically, we demonstrate that the overall NSE of a dataset is not bounded by the NSEs of all its partitions. We term this phenomenon the "Divide and Measure Nonconformity". It follows naturally from the definition of the NSE, yet because modelers often subdivide datasets in a non-random way, the resulting behavior can have unintended consequences in practice. In this note we therefore discuss the implications of the "Divide and Measure Nonconformity", examine its empirical and theoretical properties, and provide recommendations for modelers to avoid drawing misleading conclusions.

Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, and Jakob Zscheischler

Status: open (until 25 Apr 2024)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on hess-2024-59', Anonymous Referee #1, 21 Mar 2024 reply
    • AC1: 'Reply on RC1', Daniel Klotz, 12 Apr 2024 reply
  • RC2: 'HVG Comment on hess-2024-59', Hoshin Gupta, 21 Mar 2024 reply
    • AC2: 'Reply on RC2', Daniel Klotz, 12 Apr 2024 reply
  • RC3: 'Comment on hess-2024-59', Wouter Knoben, 26 Mar 2024 reply
    • AC3: 'Reply on RC3', Daniel Klotz, 12 Apr 2024 reply
Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, and Jakob Zscheischler
Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, and Jakob Zscheischler

Viewed

Total article views: 646 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
459 160 27 646 19 16
  • HTML: 459
  • PDF: 160
  • XML: 27
  • Total: 646
  • BibTeX: 19
  • EndNote: 16
Views and downloads (calculated since 29 Feb 2024)
Cumulative views and downloads (calculated since 29 Feb 2024)

Viewed (geographical distribution)

Total article views: 638 (including HTML, PDF, and XML) Thereof 638 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 24 Apr 2024
Download
Short summary
The evaluation of model performance is essential for hydrological modeling. Using performance criteria requires a deep understanding of their properties. We focus on a counterintuitive aspect of the Nash-Sutcliffe Efficiency (NSE), and show that if we divide the data into multiple parts, the overall performance can be higher than all the evaluations of the subsets. Albeit this follows from the definition of the NSE, the resulting behavior can have unintended consequences in practice.