The skill of seasonal ensemble low-flow forecasts in the Moselle River for three different hydrological models

Demirel, M. C.; Booij, M. J.; Hoekstra, A. Y.

doi:https://doi.org/10.5194/hess-19-275-2015

Articles | Volume 19, issue 1

https://doi.org/10.5194/hess-19-275-2015

© Author(s) 2015. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/hess-19-275-2015

© Author(s) 2015. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 19, issue 1

Research article

|

16 Jan 2015

Research article |

| 16 Jan 2015

The skill of seasonal ensemble low-flow forecasts in the Moselle River for three different hydrological models

M. C. Demirel, M. J. Booij, and A. Y. Hoekstra

Download

Final revised paper (published on 16 Jan 2015)
Preprint (discussion started on 23 May 2014)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC C1822: 'Interactive comment on “The skill of seasonal ensemble low flow forecasts for four different hydrological models”, by M. C. Demirel, M. J. Booij, and A. Y. Hoekstra”', Renata Romanowicz, 10 Jun 2014
- AC C1848: 'Reply to comments by Dr Renata Romanowicz', Mehmet C. Demirel, 16 Jun 2014
RC C1994: 'Review comments', Anonymous Referee #2, 23 Jun 2014
- AC C2286: 'Reply to comments by Reviewer #2', Mehmet C. Demirel, 09 Jul 2014
RC C2095: 'Interactive comment on “The skill of seasonal ensemble low flow forecasts for four different hydrological models” by M. C. Demirel et al.', Stefanie Jörg-Hess, 27 Jun 2014
- AC C2240: 'Reply to comments by Stefanie Jörg-Hess', Mehmet C. Demirel, 07 Jul 2014

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (19 Jul 2014) by Kerstin Stahl

AR by Mehmet Cüneyd Demirel on behalf of the Authors (21 Aug 2014) Author's response

ED: Referee Nomination & Report Request started (23 Sep 2014) by Kerstin Stahl

RR by Renata Romanowicz (03 Oct 2014)

RR by Anonymous Referee #2 (04 Oct 2014)

Suggestions for revision or reasons for rejection

First, I thank the authors for their detailed answer to the review comments and the revised version of their manuscript, which I found clearer. The removal of the ANN-I model makes the presentation more straightforward and results easier to interpret.
However, I still have some comments and requests for clarification, as detailed below. Minor revision is advised.

Specific comments
1. Title: I still think that the title is too general and should mention that the study is made on the Moselle basin, but it seems to be part of some disagreement with the authors.
2. L. 13: “(Precipitation P and potential evapotranspiration PET)”
3. L. 29-30: This result should be counterbalanced by the fact that ANN-E shows the poorest false alarm rate.
4. Section 2.1: As mentioned in the authors’ reply, the authors make the assumption that human influences are negligible. Although I can accept this assumption to keep the study simple, I found this is a strong assumptions and nothing shows that it is actually true. So it should be clearly mentioned in the text, since this may have some impact on the evaluation of hydrological models which simulate natural conditions.
5. Section 2.2.2: I found interesting the complementary analysis on the quality of meteorological ensembles shown in the authors’ reply. It could help interpreting results and the relative sensitivity to P and PET. Why did the authors choose not to include this information in their manuscript? There is space for that. Note that in the figures, the authors may prefer to show in the X axis the variables for the month of forecast instead of the month when the forecast was issued.
6. L. 224-225: This should be further explained here. If I understood well from the authors’ reply, the ANN-E model receives Qobs(t) as input on the time step t when the forecast is issued, and then receives the streamflow forecast of the previous time step as input for lead times larger than 1 day (Q forecast for time step t+j is used as input to forecast Q at t+j+1). This should be more clearly stated in the text, since this is not so obvious.
7. L. 241: Remove “G is groundwater” since this is not used.
8. L. 260-274: I still did not find any strong justification to support the introduction of this hybrid objective function. If this hybrid objective function is introduced, it is certainly because the authors found some added value on model performance, compared to the use of single objective functions. Nothing demonstrates this in the results shown, so I still have the same question on the usefulness of this hybrid objective function. Besides I disagree with the authors that it is not important that this hybrid function aggregates two measures which do not have the same dimension. This is a physical non-sense since it is not possible to give any dimension to MAEhybrid. This should not appear in a scientific paper. Moreover the variation ranges of MAElow and MAEinverse may be very different, which will tend to emphasize only one part of the hybrid objective function. This should be further analyzed.
9. Section 3.1.4: The authors should state in the article that all models were calibrated in simulation mode, i.e. with no update (i.e. no use of Qobs) and no forecast scenario involved. They say in their reply that this is also the case for ANN-E. However this may induce a model that gives much weight on streamflow input. Is it the case? Besides the internal weights may be different if the ANN was more classically trained in forecasting mode. Did the authors check this? The way they use the ANN may push the model towards keeping the streamflow constant (i.e. not much changing the input information), which may explain this behavior noticed for ANN to generate almost constant predictions.
10. Section 3.1.5: The authors mention later in the text (L. 511-514) that after the model updates, there may be some spin-up period. I guess this is true given their results, but it means to some extent that the way the update is done is not optimal, since one would not expect this behavior. The role of model update on model results and the possible margin of improvement in the update approaches should be further commented since updating probably plays a role in the results obtained in forecasting mode. Indeed, there is strong contrast between the poor performance of ANN-E model in validation (Figure 2b) and its apparently good/better performance shown in Fig. 6. The difference may lie in the way models are run in forecasting mode.
11. Table 4: After reading the revised version of the manuscript, I found that one benchmark test case is missing here: the case where models are run using observed P and PET as forecasts. Although this is an idealistic case, I think this would help interpreting results by giving reference performance without uncertainty on future P and PET conditions, and hence better assess the intrinsic value of the models. Could the authors add this case and shortly comment the results?
12. Section 3.2.4: Like in the case of the objective function, I did not find clear justification in the text for introducing this new evaluation metric. Which forecast quality does it reflect? Are the existing criteria not sufficient to evaluate this quality?
13. L. 368: So why is there a column for observed probabilistic flows in Table 6?
14. Table 7: The authors’ reply on CFLUX (value optimized at the upper limit) is a bit disappointing. Although I understand other studies used this variation range, I do not see strong physical reasons why CFLUX could not take values above 1 mm/d in some specific cases. What can be learnt on the model behavior from the fact that the model tries to have a high CFLUX value? Is it an issue of parameter sensitivity? Does it reveal a specific behavior of the Moselle basin?
15. Table 7: For W1 to W3, please indicate to which input they refer respectively (P, PET, Q). Since the values of weights may depend on units, were all inputs and outputs used with the same units (mm/d) to build the ANN? Given these weights are much different, what does it mean on the respective roles of these inputs on model output? Does the ANN tend to emphasize the information of one of them?
16. Figure 4b: I still did not fully understand the reason for the erratic behavior of the ANN, whose output oscillates between two values. Does not this discredit this model to some extent?
17. Table 8: Indicate in the caption for which period and lead time the performances were calculated. What bold values indicate? Please add median values also between min and max.
18. L. 506: This is not true for HR and FAR criteria, as explained a few lines later in the manuscript.
19. L. 511-514: Although this explanation seems plausible in the case of the two conceptual models, does it also apply to the ANN, which does not have internal states to update per say?
20. Fig. 6: Remind in the caption on which period the criteria are calculated (all the time steps or only low flow time steps) for each criterion.
21. L. 525, 528, 534: Is the Q95 or Q99 threshold used? Check consistency.
22. L. 554-567: I found this part not so clear. L.555-556: The results are somehow contrasted on the four evaluation criteria (results are different for the FAR). L559-560: It is not so clear that ANN-E is better than HBV given results shown in the previous sections of the manuscript. Besides, the argument given L. 565-567 tends to limit the practical usefulness of ANN-E.
23. L. 574: This somehow contradicts what is said on the objective function L. 272-273.
24. L. 587-592: The authors emphasize the fact that they proposed a new objective function and a new evaluation criterion. However, as noticed above, there is no demonstration of the usefulness of these new criteria compared to existing ones.
25. L. 608: “less skillful than the other two models”
26. L. 617-621: Given the strange behavior of the ANN-E model shown in the article (constant range in Figs. 3 and 4, erratic behavior in Figs. 4 and 5), I personally doubt this model would be really useful in practice and would convince practitioners. Therefore I wonder why the overall evaluation comes to an opposite conclusion. Is this model apparently working well for the wrong reasons?

Hide

RR by Stefanie Jörg-Hess (08 Oct 2014)

ED: Publish subject to minor revisions (Editor review) (24 Oct 2014) by Kerstin Stahl

AR by Mehmet Cüneyd Demirel on behalf of the Authors (03 Nov 2014) Author's response Manuscript

ED: Publish subject to minor revisions (Editor review) (23 Nov 2014) by Kerstin Stahl

Dear Mehmet and co-authors

Thanks for submitting your revised manuscript, which I think is now almost ready for publication.
In my assessment of your revisions, I found that some of the justified criticisms of Referee 2 that you rebutted should at least be discussed in the discussion section to give a fair assessment of the found implications to the reader. Please revise these aspects as detailed below, along with some editorial issues in order to bring this manuscript into its final, publishable, shape.

Best regards,
Kerstin

Comments:

Suggested change about unregulated flow in the Moselle:
You are modelling the Moselle and not the Rhine. The Moselle originates from the Vosges and not from the Alps. Please delete the first two sentences added on the Rhine. They are irrelevant to this study.
It is not relevant here what you assume, but what is the case and how it may influence the modelling. If you are confident that the Moselle flow you model has no flow alterations that would not be captured by the model’s dynamics, then just say so – but I suggest to say that the “alteration of flow magnitude and dynamics are minimal”, rather than to say “human influence on the river is minimal”, which is a different thing (incl. anything from urbanisation in the basin to bank stabilization etc. which is high everywhere in Europe).
However, your models are calibrated. So for the model experiment, what may matter more than general human influence is perhaps that there were no (low)flow-altering changes to regulation during the time of calibration and application. Is this the case? Tests performed on data homogeneity could prove that. Have you done any?

The referee’s valid concerns with the objective function are not properly addressed. The choice of an objective function is a crucial step in modelling and you have to remember that literature (also your paper in the future) will be used by others to decide and justify their decision. Therefore, you need to be fair and at least discuss transparently a) your decision path (based on literature and your aims) and b) all caveats encountered with you choice. This needs to be addressed with a proper discussion section to make problems and limitations very clear to the reader. Please add a fair assessment of the issues (incl the unit discrepancy) raised by the referee and based on these issues, consider toning down the conclusion (where only the further implications should be raised and not the study and results be repeated). Please revise Discussion and Conclusion accordingly.

Also, the (still very minimalistic) discussion needs to include the new finding with the apparent better forecast (higher hit rates) as a result of a more uncertain (larger range) longer-range forecast. I think this apparent better performance is a very important artefact, which actually requires to question (at least to discuss) the use of these metrics or at least what can be compared to what. And this really needs a proper discussion in order encourage follow up studies that use this paper to design their studies to address this issue better! There must be literature that has highlighted such an apparend improvement die to range previously, no? Please check and include a discussion accordingly.

Table 6. I don’t understand the answer. In addition: Table 6 needs to be changed with proper distinguishable variable names and all that text moved into table footnotes (or if the variables are explained before, then not needed) so that the reader can actually see immediately, which different counts comprise the contingency table. With all that text inside the table it is very confusing like it is. Explanations can be coordinated with the equation (maybe put this first and then use the table to explain the cases).

In all figures, the tick labels are too small and lines are too thin. In all figures there is space to make them larger. Axes labels are then sometimes excessively large, creating a very unprofessional look of the figures. Please check size requirements and conventions for sizes of labels and improve.

Table 7 comes before Table 3 in the text. The order of mentioning tables and figures in the text MUST be consistent and Tables mentioned in the text must be placed as soon as possible after the mentioning, i.e. it is not possible to refer to a later table early in the text. Needs to be changed.

Hide

AR by Mehmet Cüneyd Demirel on behalf of the Authors (02 Dec 2014) Author's response Manuscript

ED: Publish subject to technical corrections (23 Dec 2014) by Kerstin Stahl

AR by Mehmet Cüneyd Demirel on behalf of the Authors (23 Dec 2014) Author's response Manuscript

Short summary

This paper investigates the skill of 90-day low-flow forecasts using three models. From the results, it appears that all models are prone to over-predict runoff during low-flow periods using ensemble seasonal meteorological forcing. The largest range for 90-day low-flow forecasts is found for the GR4J model. Overall, the uncertainty from ensemble P forecasts has a larger effect on seasonal low-flow forecasts than the uncertainty from ensemble PET forecasts and initial model conditions.