the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Large-sample assessment of spatial scaling effects of the distributed wflow_sbm hydrological model shows that finer spatial resolution does not necessarily lead to better streamflow estimates
- 1Water Resources Section, Faculty of Civil Engineering and Geosciences, Delft University of Technology, Stevinweg 1, 2628 CN Delft, the Netherlands
- 2Netherlands eScience Center, Science Park 140, 1098 XG Amsterdam, the Netherlands
- 3Catchment and Urban Hydrology, Department of Inland Water Systems, Deltares, P.O. Box 177, 2600MH Delft, The Netherlands
- 4Operational Water Management, Department of Inland Water Systems, Deltares, P.O. Box 177, 2600MH Delft, The Netherlands
- 5Hydrology and Quantitative Water Management Group, Wageningen University and Research, P.O. Box 47, 6700AA Wageningen, The Netherlands
- 6Applied Research Center, Florida International University, FL 33174, Miami, the United States of America
- 1Water Resources Section, Faculty of Civil Engineering and Geosciences, Delft University of Technology, Stevinweg 1, 2628 CN Delft, the Netherlands
- 2Netherlands eScience Center, Science Park 140, 1098 XG Amsterdam, the Netherlands
- 3Catchment and Urban Hydrology, Department of Inland Water Systems, Deltares, P.O. Box 177, 2600MH Delft, The Netherlands
- 4Operational Water Management, Department of Inland Water Systems, Deltares, P.O. Box 177, 2600MH Delft, The Netherlands
- 5Hydrology and Quantitative Water Management Group, Wageningen University and Research, P.O. Box 47, 6700AA Wageningen, The Netherlands
- 6Applied Research Center, Florida International University, FL 33174, Miami, the United States of America
Abstract. Distributed hydrological modelling moves into the realm of hyper-resolution modelling. This results in a plethora of scaling related challenges that remain unsolved. In light of model result interpretation, finer resolution output might implicate to the user an increase in understanding of the complex interplay of heterogeneity within the hydrological system. Here we investigate spatial scaling in the realm of hyper-resolution by evaluating the streamflow estimates of the distributed wflow_sbm hydrological model based on 454 basins from the large-sample CAMELS data set. Model instances were derived at 3 spatial resolutions, namely 3 km, 1 km, and 200 m. The results show that a finer spatial resolution does not necessarily lead to better streamflow estimates at the basin outlet. Statistical testing of the objective function distributions (KGE score) of the 3 model instances show only a statistical difference between the 3 km and 200 m streamflow estimates. However, results indicate strong locality in scaling behaviour between model instances expressed by differences in KGE scores of on average 0.22. This demonstrates the presence of scaling behavior throughout the domain and indicates where locality in results is strong. The results of this study open up research paths that can investigate the changes in flux and state partitioning due to spatial scaling. This will help further understand the challenges that need to be resolved for hyper resolution hydrological modelling.
Jerom P.M. Aerts et al.
Status: final response (author comments only)
-
RC1: 'Comment on hess-2021-605', Anonymous Referee #1, 18 Jan 2022
GENERAL COMMENTS
This study is focused on the spatial resolution effect of the wflow_sbm model on the streamflow performance over the CONUS domain. The streamflow performance is evaluated through the KGE score at three spatial resolutions: 3km, 1km and 200m. To this end, the authors follow a benchmark approach in which they compare their results against a statistical benchmark in order to select or reject their simulations. The main conclusion of the study is that, besides some strong locality in the scaling behavior, finer resolutions do not implicate a better streamflow performance.
Although I find this work interesting and appropriate for the scope of this journal, there is still room for improvement before I can recommend its publication. In its current form, the manuscript should be reconsidered after major revision. I hope the comments below will help the authors improve their manuscript.
MAJOR COMMENTS
- The streamflow performance of the different model instances is evaluated through the KGE score. Although the authors state in L150-L152 that they assessed the KGE score for both a calibration and a evaluation period, it seems that the results are mainly focused on the evaluation period: the CDFs of Figure 7 correspond to the evaluation period, and at least the map in Figure 8d also corresponds to the evaluation period according to the figure caption. It is not clear if Figures 8a, b and c also correspond to the evaluation period. The calibration results briefly appear in Figure 3 for an example basin, but I consider this insufficient. Therefore, my recommendation is to include the CDFs for the calibration period in Figure 7 (see also next two comments), and clearly distinguish between calibration and evaluation scores in the figure captions.
- Similarly to the NSE score, KGE can be decomposed into three parts: the coefficient of correlation, the ratio of the mean values and the ratio of the standard deviations (Gupta et al., 2007; Knoben et al., 2019). All these CDFs should be present in the manuscript, as they will help understand why the KGE values are as they are. Apart from the CDFs for KGE, Figure 7 should collect the CDFs for these three component (not necessarily for the MARRMoT ensemble, although it would be more than welcome). These new results should be discussed as well.
- The two-fold statistical benchmark (one for the mean and one for the median) produces a poor performance (Figure 6d) that wflow_sbm can easily beat for most of the basins (Figure 6b). Although this is not a problem, I feel curious about why the KGE values are so low for the statistical benchmark. Then, the decomposition of the KGE score mentioned above should also be done for the statistical benchmark and should be incorporated into Figure 7 (a multi-panel figure where the plotted lines can be differentiated from each other may be the best way to show all this). This will help understand why the “mean statistical benchmark” outperforms the “median statistical benchmark” (Figure 6c). In particular, the ratio of the mean values will provide an interesting insight: is the ratio of the mean values closer to one for the “mean statistical benchmark”?
- The Discussion section is not structured and is written as a single block. It can be clearly divided into two parts: one part discussing the benchmark selection and one part discussing the spatial scaling effect. For sure, the new CDFs will strengthen the results and will enrich the discussion.
- I also miss in the discussion some recent and important references for the CONUS domain: for example, Mizukami et al. (2017) (already cited in the Introduction) and Rakovec et al. (2019) also carried out a large-domain calibration exercise and followed a benchmark approach to evaluate their results for the CONUS basins. Are the results of this study similar to their results?
References
Gupta et al. (2009): http://dx.doi.org/10.1016/j.jhydrol.2009.08.003
Knoben et al. (2019): https://doi.org/10.5194/hess-23-4323-2019
Mizukami et al. (2017): https://doi.org/10.1002/2017WR020401
Rakovec et al. (2019): https://doi.org/10.1029/2019JD030767
MINOR COMMENTS
Title
- The title is extremely long and sounds like a sentence extracted from the abstract or the conclusions. I would suggest a more concise title, something like “Large-sample assessment of spatial scaling effects on the streamflow estimations of a distributed hydrological model”. The reader will find that “finer spatial resolution does not necessarily lead to better streamflow estimates” in the abstract. In any case, I will leave this open to the authors.
Section 2.1.1 The CAMELS data set
- The authors point out three reasons behind failed runs: errors during parameter derivation, errors during run time and missing streamflow observations. While the last one is clear, the other two are not properly described. What do the authors mean by “errors during parameter derivation”? Is this related to the parameter estimations from external sources prior to calibration? Or is it related to the calibration procedure? On the other hand, what do the authors mean by “errors during run time”? I suggest a more detailed description.
Section 2.2.3 Model Runs & Calibration
- The parameter KsatHorFrac is the only parameter subject to calibration, and the rest of the parameters are derived from external sources. Firstly, the parameter range for KsatHorFrac should be indicated here and not in L198 when the results are presented. Secondly, it is not clear if the selection of this parameter is based on prior studies, on calibration recommendations for wflow_sbm, or on a sensitivity analysis carried out by the authors. Some information is provided in L60-L62, but I find confusing to read this in the introduction. I suggest mentioning this information in section 2.2.3 as I feel it belongs here.
- How is the model calibrated? Do the authors use a calibration algorithm? Is it based on a Montecarlo experiment? No details are given on the calibration procedure, only L153-L154 state that “the calibration procedure finds an optimal parameter value based on the KGE objective function of streamflow estimates at the basin outlet”. The calibration procedure should be properly described.
Section 2.3.2 Comparison of Streamflow Estimates
- The last sentence in L187-L188 seems incomplete, or at least has no cohesion with the previous sentence.
Section 3.4 Benchmark selection
- Instances of “Figure 7” throughout the paragraph seem to refer to Figure 6.
Section 3.5 Streamflow estimates of model instances
- “Figure 5” in L249 seems to refer to Figure 7.
- The colorbar in Figure 8c should indicate “KGE difference” or “âKGE”. “KGE value” is not correct.
Section 4 Discussion
- Should “their” in L303 be “there”?
- AC1: 'Reply on RC1', Jerom Aerts, 14 Mar 2022
-
RC2: 'Review of “Large-sample assessment of spatial scaling effects of the distributed wflow_sbm hydrological model shows that finer spatial resolution does not necessarily lead to better streamflow estimation” by Aerts et al., 2021', Shervan Gharari, 21 Feb 2022
Review of “Large-sample assessment of spatial scaling effects of the distributed wflow_sbm hydrological model shows that finer spatial resolution does not necessarily lead to better streamflow estimation” by Aerts et al., 2021
The presented manuscript is trying to evaluate the added values of modelling at finer resolutions for the streamflow simulation in a large sample hydrology framework (CAMELS data set).
I enthusiastically accepted to review the manuscript as I was interested to see the developments from the Delft team regarding hydrological modelling and infrastructure for enhancing modelling capabilities. I am a bit disappointed I should say…
In the following, are my general and specific comments:
- The manuscript fails to advance modeling infrastructure and capabilities and hydrological understanding. I think the senior co-authors can do a better job in directing and balancing coding and science for younger generations.
- It was interesting to see the mentality behind the modeling in Delft. And I am a bit puzzled why the directions are the way they are! In land surface community, which the manuscript completely misses to cover, the recent tendency is toward vector-based implementation (away from the grid-based simulation, Gharari et al., 2020, HESS, for example, advantages of vector-based setup are explained in that paper in detail). This is also true for the routing models. Additionally, the routing models and land models are more and more decoupled which means the vector-based routing models such as RAPID or mizuRoute can use runoff simulation at any (un)structured forms (grids, HRUs, GRUs, conforming or non-conforming subbasins). The land modeling community has spent a significant amount of time dealing with the upscaling of the DEM for grid-based setup. While the vector-based routing models can be simulated for any modeling resolution with underlying routing models and parameters remains identical (I mean really identical). Additionally, the gird setup results in excessive and unnecessary computational burden. If vector implementation is used, grids with similar soil, veg, and forcings are grouped, and computational costs will be significantly lower and the most optimal (explained in Gharari et al., 2020).
- I have a very bad feeling regarding the modeling resolutions the authors used! The resolutions are much finer than the actual forcing resolution used. Basically, what we see here is just the effect of forcing resampling at a finer resolution (and temperature lapsing). As the resolutions are smaller, I am afraid the actual precipitations are not very different across the scale of modeling, so the difference is coming from another source (routing perhaps?) or numerical implementation of the model.
- Following comments 2 and 3, are we just redoing a similar simulation when going to finer resolution here?
- 22 on the scale of KGE may not be that meaningful. The authors can design a simple experiment, perturb the model precipitation with a few percent, redo the simulations and see if the 0.22 gain in KGE is warranted. Or alternatively, they can use the EMDNA data set Tang et al 2021, ESSD. Personally, I am suspecting this 0.22 is well below the margin of the KGE envelop for the forcing uncertainty. I leave the streamflow uncertainty out here!
- Additionally, and following the previous comment, the margin of KGE improvement can be evaluated using methods presented by Clark et al., 2021 (cited by the authors). Why not just try?
- Back to the issue of routing with various resolutions. My understanding is that the routing parameters at various resolutions are different, although made consistently based on the work of Eilander et al., 2021. Then the setups are slightly different at various resolutions. Is that correct? If so, how the effect of this upscaling is seen on the calibrated parameters. And why line 326-328 is stated as it is!? This reinforces the use of vector-based routing in which the routing setup (network topology and its parameters) can be kept identical (even when modelling resolution or decisions are changed). Sorry if I misunderstood anything here.
- It seems that the authors have investigated lakes and reservoirs in setting up the model (I see hydrolakes). Are there any resolved lakes (on the river network) on the CAMELS data set? How significant are the lake areas within the subbasins?
- Why 454 CAMEL subbasins and not more?
- I missed how the calibration is done. Is this a single parameter calibration? I review the method section a few times but cannot comprehend it. Suggest clarifying. And sorry if I missed it.
- Back to the title, hydrology is partly about streamflow simulation. Simulation of other fluxes, states, and processes is also important. At least looking into the snow simulation for a few of the catchments might be helpful (like Figure 5 of Gharari et al., 2020 in which there are significant snow simulation differences with similar NSE for streamflow simulation). Also, the answer to the title is clear! Streamflow can be predicted with least complexity among other fluxes and states.
- Just an opinion, the authors could use an alternative model for their work. Any reasons why this model was chosen? I mean the model was not developed for small-scale hydrological applications.
- References are a bit haphazard. Before reading the manuscript, I could somehow guess much of the mentioned literature while a significant body of literature is missing. Land surface modelling community has done investigations on the effect of resolutions. There is only one citation in this work directed to those efforts (Melsen, et al., 2016). Suggest being more inclusive (which is actually very helpful for this manuscript). Is the research question relevant? Perhaps yes, but this has been discussed time and time again. I am surprised to see no references from 80s/90s that looked into streamflow simulation and non-identifiability or equifinality of parameters and model simulation configurations (essentially this is what the authors are trying to demonstrate here, the sensitivity of streamflow simulation to various model setup configurations). The introduction talks about MPR and self-organization, while to what I see none of those principles were used. Sometimes references are given without reflection and completeness. For example, the choice with the deficiency of objective function. The author can mention many more relevant studies. For example, Gharari et al., 2013 is very relevant here and one of the papers mentioned here (lines 351 to 360) is a specific and simplified version of that work.
- Are we looking at the scaling or just a change in modelling resolution? Scaling is more about collective behaviours at a given scale (and hopefully understanding or explaining it). Change of resolution is only a model application at different configurations. Any explanation on that?
- And final comment, I am a bit concerned about using off the shelves material and packages. I think there is a line between being a user or a developer. Are you fully confident in all the bits and pieces of your workflow in setting up the model? for example ESMValTool. It seems to be a very good package however isn’t it limiting in sense of forcing preparation? Why not use a more as more data agnostic package such as EASYMORE or more elaborated ESMF? Maybe my knowledge of EMSValTool is limited but the first look it doesn’t seem to be designed for forcing preparation for models primarily. Another question, why move to Julia and not go to Fortran for speed? The speed is not comparable to python for example. Also, Fortran commands are more stable over time compared to Python packages which makes maintenances much easier (a pandas commands might change in future releases).
Overall, I think the manuscript needs a major-major revision. Hope my comments are useful and hope the authors reflect on my comments. Reflection is rather missing in today’s hydrology I would say. Also, don’t hesitate to contact me personally if any questions arise and sorry for the delay in review process.
With regards,
Shervan Gharari
20th February 2022, Saskatoon
- AC2: 'Reply on RC2', Jerom Aerts, 14 Mar 2022
Jerom P.M. Aerts et al.
Jerom P.M. Aerts et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 546 | 166 | 15 | 727 | 6 | 10 |
- HTML: 546
- PDF: 166
- XML: 15
- Total: 727
- BibTeX: 6
- EndNote: 10
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1