Articles | Volume 28, issue 15
https://doi.org/10.5194/hess-28-3665-2024
https://doi.org/10.5194/hess-28-3665-2024
Technical note
 | 
13 Aug 2024
Technical note |  | 13 Aug 2024

Technical Note: The divide and measure nonconformity – how metrics can mislead when we evaluate on different data partitions

Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, and Jakob Zscheischler

Related authors

Unveiling the Limits of Deep Learning Models in Hydrological Extrapolation Tasks
Sanika Baste, Daniel Klotz, Eduardo Acuña Espinoza, Andras Bardossy, and Ralf Loritz
EGUsphere, https://doi.org/10.5194/egusphere-2025-425,https://doi.org/10.5194/egusphere-2025-425, 2025
This preprint is open for discussion and under review for Hydrology and Earth System Sciences (HESS).
Short summary
EARLS: A runoff reconstruction dataset for Europe
Daniel Klotz, Peter Miersch, Thiago V. M. do Nascimento, Fabrizio Fenicia, Martin Gauch, and Jakob Zscheischler
Earth Syst. Sci. Data Discuss., https://doi.org/10.5194/essd-2024-450,https://doi.org/10.5194/essd-2024-450, 2025
Preprint under review for ESSD
Short summary
Technical note: An approach for handling multiple temporal frequencies with different input dimensions using a single LSTM cell
Eduardo Acuña Espinoza, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Ralf Loritz, and Uwe Ehret
EGUsphere, https://doi.org/10.5194/egusphere-2024-3355,https://doi.org/10.5194/egusphere-2024-3355, 2024
Short summary
HESS Opinions: Never train a Long Short-Term Memory (LSTM) network on a single basin
Frederik Kratzert, Martin Gauch, Daniel Klotz, and Grey Nearing
Hydrol. Earth Syst. Sci., 28, 4187–4201, https://doi.org/10.5194/hess-28-4187-2024,https://doi.org/10.5194/hess-28-4187-2024, 2024
Short summary
A data-centric perspective on the information needed for hydrological uncertainty predictions
Andreas Auer, Martin Gauch, Frederik Kratzert, Grey Nearing, Sepp Hochreiter, and Daniel Klotz
Hydrol. Earth Syst. Sci., 28, 4099–4126, https://doi.org/10.5194/hess-28-4099-2024,https://doi.org/10.5194/hess-28-4099-2024, 2024
Short summary

Related subject area

Subject: Catchment hydrology | Techniques and Approaches: Theory development
Can system dynamics explain long-term hydrological behaviors? The role of endogenous linking structure
Xinyao Zhou, Zhuping Sheng, Kiril Manevski, Rongtian Zhao, Qingzhou Zhang, Yanmin Yang, Shumin Han, Jinghong Liu, and Yonghui Yang
Hydrol. Earth Syst. Sci., 29, 159–177, https://doi.org/10.5194/hess-29-159-2025,https://doi.org/10.5194/hess-29-159-2025, 2025
Short summary
Characterizing nonlinear, nonstationary, and heterogeneous hydrologic behavior using ensemble rainfall–runoff analysis (ERRA): proof of concept
James W. Kirchner
Hydrol. Earth Syst. Sci., 28, 4427–4454, https://doi.org/10.5194/hess-28-4427-2024,https://doi.org/10.5194/hess-28-4427-2024, 2024
Short summary
Annual memory in the terrestrial water cycle
Wouter R. Berghuijs, Ross A. Woods, Bailey J. Anderson, Anna Luisa Hemshorn de Sánchez, and Markus Hrachowitz
EGUsphere, https://doi.org/10.5194/egusphere-2024-2954,https://doi.org/10.5194/egusphere-2024-2954, 2024
Short summary
Ratio limits of water storage and outflow in a rainfall–runoff process
Yulong Zhu, Yang Zhou, Xiaorong Xu, Changqing Meng, and Yuankun Wang
Hydrol. Earth Syst. Sci., 28, 4251–4261, https://doi.org/10.5194/hess-28-4251-2024,https://doi.org/10.5194/hess-28-4251-2024, 2024
Short summary
Bimodal hydrographs in a semi-humid forested watershed: characteristics and occurrence conditions
Zhen Cui, Fuqiang Tian, Zilong Zhao, Zitong Xu, Yongjie Duan, Jie Wen, and Mohd Yawar Ali Khan
Hydrol. Earth Syst. Sci., 28, 3613–3632, https://doi.org/10.5194/hess-28-3613-2024,https://doi.org/10.5194/hess-28-3613-2024, 2024
Short summary

Cited articles

Addor, N., Newman, A. J., Mizukami, N., and Clark, M. P.: The CAMELS data set: catchment attributes and meteorology for large-sample studies, Hydrol. Earth Syst. Sci., 21, 5293–5313, https://doi.org/10.5194/hess-21-5293-2017, 2017. a, b
Beven, K.: Benchmarking hydrological models for an uncertain future, Hydrol. Process., 37, e14882, https://doi.org/10.1002/hyp.14882, 2023. a
Clark, M. P., Vogel, R. M., Lamontagne, J. R., Mizukami, N., Knoben, W. J., Tang, G., Gharari, S., Freer, J. E., Whitfield, P. H., Shook, K. R., and Papalexiou, S. M.: The abuse of popular performance metrics in hydrologic modeling, Water Resour. Res., 57, e2020WR029001, https://doi.org/10.1029/2020WR029001, 2021. a, b, c, d, e, f
Duc, L. and Sawada, Y.: A signal-processing-based interpretation of the Nash–Sutcliffe efficiency, Hydrol. Earth Syst. Sci., 27, 1827–1839, https://doi.org/10.5194/hess-27-1827-2023, 2023. a
Feng, D., Beck, H., Lawson, K., and Shen, C.: The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment, Hydrol. Earth Syst. Sci., 27, 2357–2373, https://doi.org/10.5194/hess-27-2357-2023, 2023. a
Download
Short summary
The evaluation of model performance is essential for hydrological modeling. Using performance criteria requires a deep understanding of their properties. We focus on a counterintuitive aspect of the Nash–Sutcliffe efficiency (NSE) and show that if we divide the data into multiple parts, the overall performance can be higher than all the evaluations of the subsets. Although this follows from the definition of the NSE, the resulting behavior can have unintended consequences in practice.
Share