It is somewhat depressing that after all the decades of past work on aleatory and epistemic uncertainties in both data, models and identification of parameters, there are still studies that essentially ignore the impacts (except for a conclusion that parameter uncertainty is not an issue because different random seeds give similar optimal parameter sets for some OFs).    

But why have you not considered that  some of the data you are using might be disinformative for model evaluation; that there may be parameter sets close to your optima that will give similar "performance" however that is measured; that different periods of data (with different errors) will give different optimal parameter sets, etc etc (see for example Beven, K. J., 2024, A short history of philosophies of hydrological model evaluation and hypothesis testing, WIRES Water, e1761, 69 (5): 519-527, https://doi.org/10.1002/wat2.1761 and the references therein).

We have known these things for a very long time - but the real issue to be addressed is whether a model (even when optimised as in this study) can really be considered as fit for purpose when there are often glaring visual issues in performance (during wetting up periods at the end of summer for example)  that are glossed over by the types of global OFs used here).   That was one of the reasons why I rejected the concept of optimal parameter sets more than 30 years ago now in favour of seeking models that might be consistent with the observations and what we know about their uncertainities.   Trying to assess those uncertainties is, of course, a much more difficult problem than simply applying an optimisation algorithm (particularly for the epistemic uncertainties), but just thinking about what might be involved in doing so is a really valuable exercise.

Apologies in advance for this little rant but if we do not approach the modelling process with a bit deeper thought, how are we going to progress the science?    That surely requires ways of rejecting models and then trying to do better, not of accepting that an optimised model is de facto considered satisfactory.  

Keith Beven   

The authors, three of four affiliated with the University of Calgary, Canada, need to explain a glaring omission in references of Melsen et al. (2025). (https://doi.org/10.1080/02626667.2025.2475105).  Is it simply the study, on which the discussion paper is based, was substantially completed before the publication of the Melsen et al. paper?

I've been both amused and amazed, a long lost but recently found Nash-Ding efficiency (NDE; Ding, 1974, Page 65) performs as well as a late comer, the KGE, even numerically, Duc and Sawada (2023, Figure 2). (https://doi.org/10.5194/hess-27-1827-2023).

I'm grateful to both my alma mater, the University of Guelph, Canada, and my former employer, Ontario Ministry of Natural Resources in Toronto. Because of their foresight, I found myself, one of three young Government of Ontario engineers, listening to Master Nash delivering his  NSE lecture at Guelph, one splendid fall afternoon of 1968.

Whose idea it was to invite Nash of Wallingford, UK, and/or Galway, Ireland, to visit Guelph? Their identity remains buried in archives. The search continues.

PS. This reminiscence has been prompted upon reading the 'little rant' in the last paragraph raised by Keith Beven in CC1 elsewhere.

Let me start my comment in an unusual way: Imagine yourself in the middle of the difficult process of choosing one or maximum two metrics for model calibration within a wider model intercomparison project, involving several modelling groups using different models. The project wants to renew national-scale climate change impact simulations because the existing ones show deficiencies with respect to low flow simulations. You have your own preference for calibration metrics ("I only use NSE because at least I know very well what it does and it has a clear link to formal parameter estimation literature"), but you have to discuss with colleagues who have very different preferences ("KGE-like metrics are the new standard", "we should use the new metric by xxx").  After several discussion rounds, you discover a paper that investigates exactly the question that you have been ruminating for several weeks: Is there a modern hydrological metric that outperforms all others across a typical range of signatures for typical hydrological regimes? What a chance that someone took the time to analyze this question! Many thanks to the authors of this study for having provided a detailed analysis of a specific modelling question that many of us are facing. The design of the study emphasizes an aspect that is often omitted: the variability arising from different model structures, which is often lacking from available studies, in particular in times of large-ensemble hydrology. We have become so excited about using hundreds of catchments for our studies that we tend to omit model ensembles.

As pointed out by other comments in this discussion, the study does not explicitly address data or parameter uncertainty: every data point included in the analysis (a signature value obtained for a given model calibrated with a given objective function for a given catchment) could in principle be augmented by analyzing e.g. the 20 best calibrations or by analyzing different input data sets or by analyzing different data portions. But this would be a different study.

The study does also not put a lot of emphasis on connecting the results to dominant hydrological processes or controls. In particular, it would be nice to see some discussion of what signatures are (by nature!) dominated by the input data. I could think of rising limb density, which can arguably only be well simulated if the rainfall input has good properties. An attempt is made in Table 5 where the signatures are put into categories, but the category “streamflow” is perhaps a bit too generic. How do these signatures depend on input data quality and the climate, hillslope-scale partionning functions or storage-release functions ? The ones that strongly depend on "functions" are probably more strongly influenced by objective function choice. Any thoughts on this could perhaps complement the discussion.

Two detailed comments:

• The term « hydrological regime » is used at instances e.g. in 83 where I believe it should read the “hydrological response” or “hydrological behaviour” since the signatures do not only capture the regime
• Sentence at line 463 ("The claim that KGE resolves (..)") seems incomplete.