the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Seamless streamflow model provides forecasts at all scales from daily to monthly and matches the performance of non-seamless monthly model
- 1School of Civil, Environmental and Mining Engineering, University of Adelaide, SA, Australia
- 2Bureau of Meteorology, ACT, Canberra, Australia
- 3Bureau of Meteorology, VIC, Melbourne, Australia
- 4School of Engineering, University of Newcastle, Callaghan, NSW, Australia
- 1School of Civil, Environmental and Mining Engineering, University of Adelaide, SA, Australia
- 2Bureau of Meteorology, ACT, Canberra, Australia
- 3Bureau of Meteorology, VIC, Melbourne, Australia
- 4School of Engineering, University of Newcastle, Callaghan, NSW, Australia
Abstract. Subseasonal streamflow forecasts inform a multitude of water management decisions, from early flood warning to reservoir operation. ‘Seamless’ forecasts, i.e., forecasts that are reliable over a range of lead times (1–30 days) and when aggregated to multiples time scales (e.g. daily and monthly) are of clear practical interest. However, existing forecasting products are often ‘non-seamless’, i.e., designed for a single time scale and lead time (e.g. 1 month ahead). If seamless forecasts are to be a viable replacement for existing ‘non-seamless’ forecasts, it is important that they offer (at least) similar predictive performance at the time scale of the non-seamless forecast.
This study compares the recently developed seamless daily Multi-Temporal Hydrological Residual Error (MuTHRE) model to the (non-seamless) monthly streamflow post-processing (QPP) model that was used in the Australian Bureau of Meteorology’s Dynamic Forecasting System. Streamflow forecasts from both models are generated for 11 Australian catchments, using the GR4J hydrological model and post-processed rainfall forecasts from the ACCESS-S climate model. Evaluating monthly forecasts with key performance metrics (reliability, sharpness, bias and CRPS skill score), we find that the seamless MuTHRE model provides essentially the same performance as the non-seamless monthly QPP model for the vast majority of metrics and temporal stratifications (months and years). When this outcome is combined with the numerous practical benefits of seamless forecasts it is clear that seamless forecasting technologies, such as the MuTHRE model, are not only viable, but a preferred choice for future research development and practical adoption of streamflow forecasting.
David McInerney et al.
Status: final response (author comments only)
-
RC1: 'Comment on hess-2021-589', Anonymous Referee #1, 02 May 2022
I found this manuscript to be of high quality, interesting and very well written. In particular, I find the figures very clear and of high quality. I have a few minor comments that I would like to see addressed, but this should not require a lot of time.
1) When I first started reading the manuscript, including the title, I was very confused by what you refer to when you talk about a "model". To me, and I think for most hydrologists, our first reflex is to think about the hydrological model (GR4J in your case). It later became clear that you were most of the time refering to the post-processing method, but I was still bothered throughout the paper by the fact that it could be clearer. For one thing, I would suggest changing the title to indicate that you are interested by comparing two post-processing methods. Second, I would avoid the use of the QPP acronym and instead refer to the "post-processing method" or "post-processing model". I recongnize that it would be a bit longer, but in my opinion it would be clearer. I'm one of those persons who don't like acronyms too much... Section 2 should be named "Post-processing methods" or "Post-processing models", and maybe the titles of the subsections could be adjusted accordingly. Finally, I would suggest mentioning GR4J explicitly in section 2.1 and refering to section 3.2 for greater details. In fact, I was initially a bit bothered by the fact that GR4J was described in the case study and not in the "Forecasting models" section, because to me, GR4J is a model and it can be used jointly with meteorological forecasts to produce streamflow forecasts. I think I understand what you wanted to do here: you wanted to avoid drawing too much attention to GR4J because it is not the central element of your study. So in the end, I don't mind too much GR4J being described in section 3.2, but I really think it would help clarify/distinguish things if you could mention it in section 2.1 too.
2) This is linked to point 1. When I read the abstract and intro, I had a bit of difficulty understanding what you meant by "designed" for a single time step", because I had, again, a hydrological model in mind. So I was think that, for instance, if GR4J runs at a daily time step, there is nothing stoping you from aggregating it at the monthly time step. So I wasn't completely sure were you were going with that, until I read section 2. It then became very clear that the monthly post-processing method is completely inapplicable at the daily time step and I understood what you ment in the abstract and intro. I think it would be better it could be clearer right from the start, and I think that making it more explicit that you refer to post-processing methods would really help.
3) Page 1 line 20: Would it be more accurate to refer to ACCESS-S as an atmospheric model? When I think of a climate model, I think about the ones used for climate change study, that really model the climate over long periods and for which the fluctuations at smaller time steps have no real significance. I know they are basically very similar in their structures, but if I understand correctly, they are not run in the same way at all
4) I was really thankful for Figure 1. Section 2 was a bit heavy, but this figure is really helpfull to understand how everything is put together. Great work. Very small point: is it possible that the accronym REM is not defined in the text? Maybe I missed it? I understand it means Residual Error Model but can you please make sure it is defined in the text when first mentionned? Just for clarity.
5) Very very small point: Line 111, why chose the subscript "foc"? Why not "fcst" if it is forecasted? I think it would be clearer?
6) Page 5 line 141: do you mean each individual member or is it really each forecast? I understand you reduce (or "collapse") the forecasts to deterministic ones, so I guess it is really "each forecast", but I'm not sure. Can you maybe clarify? I was a bit lost.
7) Page 6 line 167: I like that you use the word "collapse" when refering to the transition from ensemble to deterministic. I think I will adopt this terminology myself in the future.
8) Section 2.4 is very helpful. Very clear. Thank you.
9) Page 10 line 258: I would suggest referring to Schepen et al (2018) as a pre-processing method instead of a post-processing method, especially in the context of your study, since you use the result from this method as an input.
10) I like section 3.4.1. The choice of performance indicators is very well justified and they are clearly appropriate.
11) I was surprised that Figure 5 b was not discussed more, especially in relations to the findings in the next page (statistical significance for the different metrics. When looking at Figure 5b, I can't help but think that the monthly post-processing method is not doing a very good job, and this is a bit surprising given that it is trained for monthly forecasts. First, there is an underestimation of streamflow between 2000-2001, and then (2006-2008) a pretty large overestimation. Why is that? Any idea? Then later we learn that, according to the different performance metrics, the differences in performance are actually not often statistically significant. I was surprised by that, because of that figure. I would really appreciate a bit more discussion/explanation of what is happening in 2000-2001 and 2006-2008.
12) Line 388: I suggest removing the word "actually" to avoid repetition with the previous sentence
- AC1: 'Reply on RC1', David McInerney, 02 Jun 2022
-
RC2: 'Comment on hess-2021-589', Anonymous Referee #2, 04 May 2022
Summary
In this paper, the authors introduce a Multi-Temporal Hydrological Residual Error (MuTHRE) model that enables the production of seamless streamflow forecasts (e.g., daily, weekly, fortnightly, monthly) within the range of 1-30 days. The approach is described and compared against a non-seamless streamflow post-processing (QPP) model implemented by the Australian Bureau of Meteorology’s Dynamic Forecasting System. The comparison is performed in 11 Australian catchments in terms of several forecast attributes, and the authors conclude that the MuTHRE model is not only capable of providing good performance for daily streamflow forecasts and cumulative volumes, but also similar performance to that obtained with the non-seamless QPP model for monthly flows.
Overall, this is an interesting manuscript that contributes with encouraging results on the use of seamless streamflow forecasting frameworks. The motivation is clearly stated and the results are nicely presented. There is, nevertheless, a lot of room for improving explanations of the model formulation, streamflow forecast generation, and verification, so that any reader could reproduce the results presented here. There are other minor comments and editorial suggestions that may also help the authors to improve the quality of their manuscript.
Main comment:
1. Model description (section 2): I found this section very hard to understand. I think the manuscript would greatly benefit from re-organizing the material and improving definitions and descriptions. For example:
- It seems that the two approaches compared here follow the same general model structure (equations 1 and 2). Is that what you mean with “both QPP models”? Can you please be more explicit? Also, Qt is described as a “probability model for streamflow” (equation 1), and then as a “residual error model” (L103, equation 2) when it is, in reality, the sum of deterministic model output and a residual error term. I wonder if you actually need equation (1) in this description.
- I think it would be better to have the information presented in L191-214 (differences between MuTHRE and monthly QPP model) right after section 2.1. The authors should consider separating Figure 1 (which is very nice) into two figures: one for model structure (which could include model equations for more direct comparisons between model structures), and another figure for model calibration and forecasting.
- The meaning of z should be included after presenting equation (2) (perhaps in line 107).
- Since Xt is also used to describe state variables in the hydrology literature (especially in data assimilation books/papers), I think ut would be more appropriate for meteorological forcings (e.g., Liu and Gupta 2007). Additionally, in L125 you describe st as a time-varying scaling factor, while the same variable is used to describe hydrological model states in L100.
- L113: I presume that the raw streamflow forecasts do not account for uncertainty in hydrologic model parameters. Can you please clarify?
- L135: Is the ensemble size still Nfoc after adding the residual term?
- L141: What do you mean by “individual raw forecast”? Each ensemble member produced with the ensemble of rainfall forecasts?
- L148: how is m* determined?
- Since the paper should be self-contained, additional information on the calibration procedures referred to in L162 and L190 should be provided (what are the calibration period, objective functions, and optimization algorithms?). A couple of sentences should suffice.
- L167: “and then collapsed to a deterministic forecast by taking the median”. Is this current operational practice?
- L111, L112, L135, L168, L169, and elsewhere: is “replicate” the same as “ensemble member”?
Additional minor comments
2. L33-35: It makes more sense to me to describe common practice before referring to the need for seamless forecasts. Also, it would be worth highlighting that non-seamless forecasting efforts have been (and are being) conducted in South America (e.g., Souza Filho and Lall 2003; Mendoza et al. 2014), Europe (e.g., Ionita et al. 2008; Hidalgo-Muñoz et al. 2015), Asia (e.g., Pal et al. 2013) and everywhere else around the world, with appropriate citations.
3. L37: “This is the focus of our study”. This reads out of place here. I recommend deleting this sentence or moving it toward the end of the introduction.
4. Figure 2: How many values are contained in each boxplot? One per basin? Since you have only 11 catchments, I think it would be better to show one line per basin. Further, it would be informative for readers to have a table with the name of the station, basin-averaged elevation, area, mean annual runoff, mean annual precipitation, mean annual temperature, annual runoff ratio, and aridity index.
5. L273: Are you working with calendar years or water years? Are daily forecasts produced each day in year j with MuTHRE, or only at the beginning of each month? What is the final ensemble size of your forecasts?
6. L274-275: The problem of hydrologic memory in Australian catchments and its implications for cross-validation has been previously documented (e.g., Robertson et al. 2013; Pokhrel et al. 2013). I recommend the authors read and cite these papers here. The following blog article is also relevant: https://hepex.inrae.fr/how-good-is-my-forecasting-method-some-thoughts-on-forecast-evaluation-using-cross-validation-based-on-australian-experiences/
7. L292: Perhaps it would be better to replace the word "uncertainty" with "spread". Also, it would be informative to state that sharpness is a forecast attribute only (i.e., it does not depend on the observations).
8. L296: Since CRPS measures the difference between forecast and observation CDFs, it would be better to refer to “probability forecast errors” instead of “combined performance”.
9. Figure 4: How are confidence limits generated? Do you compute the metric merging forecasts from all basins? Please clarify these points in the manuscript.
10. L368-370: You mention that reliability results are similar, although the boxplots look different. I recommend applying a statistical test to determine whether the distributions of these metrics are significantly different.
11. L370 and elsewhere: “practically significant” or “significant”. Are the authors referring to a statistically significant result? If not, I suggest re-wording or deleting the word ‘significant’.
12. Figure 6: I think you should say "overall monthly performance" in the caption, and perhaps remind readers here what "overall" means. Are you grouping the results of all basins? In the left panels, how many points are contained in each boxplot? In the center and right panels, how are the confidence limits computed?
13. L421: Shall we expect persistence in rainfall, given the chaotic nature of the atmosphere?
14. L426: I encourage the authors to replace the last sentence of this paragraph (which reads a lot like "propaganda") with a more quantitative statement regarding the performance of MuTHRE.
15. L467: “This was not feasible in this study”. If you cannot provide an explanation on why was not feasible, I suggest deleting this sentence.
16. L479: “High-quality forecasts”. Note that the quality depends a lot on the forecast attributes you are analyzing. I think it would be good to provide a brief discussion (maybe in section 5) about tradeoffs between the metrics included here (e.g., how your forecast system can improve reliability at the cost of losing sharpness), and what makes a forecast “good” or “high-quality”.
Suggested edits
17. L51: ‘Hydro-electric’ -> ‘hydropower’.
18. L70: ‘drop in’ -> ‘loss of’.
19. L312: ‘which’ -> ‘who’.
20. L377: ‘in 1 month (September)’ -> ‘in September’.
21. L380: delete ‘similar/better performance in all months, with practically’.
22. L382: delete ‘similar/better performance in 19 out of 22 years, with practical’.
23. L451: ‘Simplifies’ -> ‘A simplified’.
24. L473: ‘to forecasts’ -> ‘compared to forecasts’.
References
Hidalgo-Muñoz, J. M., S. R. Gámiz-Fortis, Y. Castro-Díez, D. Argüeso, and M. J. Esteban-Parra, 2015: Long-range seasonal streamflow forecasting over the Iberian Peninsula using large-scale atmospheric and oceanic information. Water Resour. Res., 51, 3543–3567, doi:10.1002/2014WR016826.
Ionita, M., G. Lohmann, and N. Rimbu, 2008: Prediction of Spring Elbe Discharge Based on Stable Teleconnections with Winter Global Temperature and Precipitation. J. Clim., 21, 6215–6226, doi:10.1175/2008JCLI2248.1. http://journals.ametsoc.org/doi/abs/10.1175/2008JCLI2248.1 (Accessed February 18, 2015).
Liu, Y., and H. V. Gupta, 2007: Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework. Water Resour. Res., 43, W07401, doi:10.1029/2006WR005756.
Mendoza, P. A., B. Rajagopalan, M. P. Clark, G. Cortés, and J. McPhee, 2014: A robust multimodel framework for ensemble seasonal hydroclimatic forecasts. Water Resour. Res., 50, 6030–6052, doi:10.1002/2014WR015426.
Pal, I., U. Lall, a. W. Robertson, M. a. Cane, and R. Bansal, 2013: Predictability of Western Himalayan river flow: melt seasonal inflow into Bhakra Reservoir in northern India. Hydrol. Earth Syst. Sci., 17, 2131–2146, doi:10.5194/hess-17-2131-2013. http://www.hydrol-earth-syst-sci.net/17/2131/2013/.
Pokhrel, P., Q. J. Wang, and D. E. Robertson, 2013: The value of model averaging and dynamical climate model predictions for improving statistical seasonal streamflow forecasts over Australia. Water Resour. Res., 49, 6671–6687, doi:10.1002/wrcr.20449.
Robertson, D. E., P. Pokhrel, and Q. J. Wang, 2013: Improving statistical forecasts of seasonal streamflows using hydrological model output. Hydrol. Earth Syst. Sci., 17, 579–593, doi:10.5194/hess-17-579-2013.
Souza Filho, F. A., and U. Lall, 2003: Seasonal to interannual ensemble streamflow forecasts for Ceara, Brazil: Applications of a multivariate, semiparametric algorithm. Water Resour. Res., 39, 1307, doi:10.1029/2002WR001373.
- AC2: 'Reply on RC2', David McInerney, 02 Jun 2022
David McInerney et al.
David McInerney et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 400 | 108 | 21 | 529 | 11 | 7 |
- HTML: 400
- PDF: 108
- XML: 21
- Total: 529
- BibTeX: 11
- EndNote: 7
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1