Articles | Volume 28, issue 15
https://doi.org/10.5194/hess-28-3665-2024
https://doi.org/10.5194/hess-28-3665-2024
Technical note
 | 
13 Aug 2024
Technical note |  | 13 Aug 2024

Technical Note: The divide and measure nonconformity – how metrics can mislead when we evaluate on different data partitions

Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, and Jakob Zscheischler

Related authors

Analyzing the generalization capabilities of hybrid hydrological models for extrapolation to extreme events
Eduardo Acuna Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Nicole Bäuerle, and Uwe Ehret
EGUsphere, https://doi.org/10.5194/egusphere-2024-2147,https://doi.org/10.5194/egusphere-2024-2147, 2024
Short summary
A data-centric perspective on the information needed for hydrological uncertainty predictions
Andreas Auer, Martin Gauch, Frederik Kratzert, Grey Nearing, Sepp Hochreiter, and Daniel Klotz
Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2024-64,https://doi.org/10.5194/hess-2024-64, 2024
Revised manuscript accepted for HESS
Short summary
HESS Opinions: Never train an LSTM on a single basin
Frederik Kratzert, Martin Gauch, Daniel Klotz, and Grey Nearing
Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2023-275,https://doi.org/10.5194/hess-2023-275, 2024
Revised manuscript accepted for HESS
Short summary
Technical note: Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks
Grey S. Nearing, Daniel Klotz, Jonathan M. Frame, Martin Gauch, Oren Gilon, Frederik Kratzert, Alden Keefe Sampson, Guy Shalev, and Sella Nevo
Hydrol. Earth Syst. Sci., 26, 5493–5513, https://doi.org/10.5194/hess-26-5493-2022,https://doi.org/10.5194/hess-26-5493-2022, 2022
Short summary
The Great Lakes Runoff Intercomparison Project Phase 4: the Great Lakes (GRIP-GL)
Juliane Mai, Hongren Shen, Bryan A. Tolson, Étienne Gaborit, Richard Arsenault, James R. Craig, Vincent Fortin, Lauren M. Fry, Martin Gauch, Daniel Klotz, Frederik Kratzert, Nicole O'Brien, Daniel G. Princz, Sinan Rasiya Koya, Tirthankar Roy, Frank Seglenieks, Narayan K. Shrestha, André G. T. Temgoua, Vincent Vionnet, and Jonathan W. Waddell
Hydrol. Earth Syst. Sci., 26, 3537–3572, https://doi.org/10.5194/hess-26-3537-2022,https://doi.org/10.5194/hess-26-3537-2022, 2022
Short summary

Related subject area

Subject: Catchment hydrology | Techniques and Approaches: Theory development
Bimodal hydrographs in a semi-humid forested watershed: characteristics and occurrence conditions
Zhen Cui, Fuqiang Tian, Zilong Zhao, Zitong Xu, Yongjie Duan, Jie Wen, and Mohd Yawar Ali Khan
Hydrol. Earth Syst. Sci., 28, 3613–3632, https://doi.org/10.5194/hess-28-3613-2024,https://doi.org/10.5194/hess-28-3613-2024, 2024
Short summary
Flood drivers and trends: a case study of the Geul River catchment (the Netherlands) over the past half century
Athanasios Tsiokanos, Martine Rutten, Ruud J. van der Ent, and Remko Uijlenhoet
Hydrol. Earth Syst. Sci., 28, 3327–3345, https://doi.org/10.5194/hess-28-3327-2024,https://doi.org/10.5194/hess-28-3327-2024, 2024
Short summary
Power law between the apparent drainage density and the pruning area
Soohyun Yang, Kwanghun Choi, and Kyungrock Paik
Hydrol. Earth Syst. Sci., 28, 3119–3132, https://doi.org/10.5194/hess-28-3119-2024,https://doi.org/10.5194/hess-28-3119-2024, 2024
Short summary
Characterizing nonlinear, nonstationary, and heterogeneous hydrologic behavior using Ensemble Rainfall-Runoff Analysis (ERRA): proof of concept
James W. Kirchner
Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2024-103,https://doi.org/10.5194/hess-2024-103, 2024
Short summary
Stream water sourcing from high-elevation snowpack inferred from stable isotopes of water: a novel application of d-excess values
Matthias Sprenger, Rosemary W. H. Carroll, David Marchetti, Carleton Bern, Harsh Beria, Wendy Brown, Alexander Newman, Curtis Beutler, and Kenneth H. Williams
Hydrol. Earth Syst. Sci., 28, 1711–1723, https://doi.org/10.5194/hess-28-1711-2024,https://doi.org/10.5194/hess-28-1711-2024, 2024
Short summary

Cited articles

Addor, N., Newman, A. J., Mizukami, N., and Clark, M. P.: The CAMELS data set: catchment attributes and meteorology for large-sample studies, Hydrol. Earth Syst. Sci., 21, 5293–5313, https://doi.org/10.5194/hess-21-5293-2017, 2017. a, b
Beven, K.: Benchmarking hydrological models for an uncertain future, Hydrol. Process., 37, e14882, https://doi.org/10.1002/hyp.14882, 2023. a
Clark, M. P., Vogel, R. M., Lamontagne, J. R., Mizukami, N., Knoben, W. J., Tang, G., Gharari, S., Freer, J. E., Whitfield, P. H., Shook, K. R., and Papalexiou, S. M.: The abuse of popular performance metrics in hydrologic modeling, Water Resour. Res., 57, e2020WR029001, https://doi.org/10.1029/2020WR029001, 2021. a, b, c, d, e, f
Duc, L. and Sawada, Y.: A signal-processing-based interpretation of the Nash–Sutcliffe efficiency, Hydrol. Earth Syst. Sci., 27, 1827–1839, https://doi.org/10.5194/hess-27-1827-2023, 2023. a
Feng, D., Beck, H., Lawson, K., and Shen, C.: The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment, Hydrol. Earth Syst. Sci., 27, 2357–2373, https://doi.org/10.5194/hess-27-2357-2023, 2023. a
Download
Short summary
The evaluation of model performance is essential for hydrological modeling. Using performance criteria requires a deep understanding of their properties. We focus on a counterintuitive aspect of the Nash–Sutcliffe efficiency (NSE) and show that if we divide the data into multiple parts, the overall performance can be higher than all the evaluations of the subsets. Although this follows from the definition of the NSE, the resulting behavior can have unintended consequences in practice.