Daily ensemble river discharge reforecasts and real-time forecasts from the operational Global Flood Awareness System

Harrigan, Shaun; Zsoter, Ervin; Cloke, Hannah; Salamon, Peter; Prudhomme, Christel

doi:https://doi.org/10.5194/hess-27-1-2023

Articles | Volume 27, issue 1

https://doi.org/10.5194/hess-27-1-2023

© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-27-1-2023

© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 27, issue 1

Research article

| Highlight paper

|

02 Jan 2023

Research article | Highlight paper |

| 02 Jan 2023

Daily ensemble river discharge reforecasts and real-time forecasts from the operational Global Flood Awareness System

Shaun Harrigan, Ervin Zsoter, Hannah Cloke, Peter Salamon, and Christel Prudhomme

Download

Final revised paper (published on 02 Jan 2023)
Supplement to the final revised paper
Preprint (discussion started on 16 Oct 2020)
Supplement to the preprint

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Important update on a streamflow forecasting system of global significance', Anonymous Referee #1, 22 Nov 2020
- AC1: 'Initial response to RC1', Shaun Harrigan, 08 Dec 2020
RC2: 'Referee comments on Glofas in C3S', Berit Arheimer, 17 Dec 2020
- AC2: 'Initial response to RC2', Shaun Harrigan, 20 Jan 2021
RC3: 'Referree comment', Anonymous Referee #3, 23 Dec 2020
- AC3: 'Initial response to RC3', Shaun Harrigan, 20 Jan 2021

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (further review by editor and referees) (22 Jan 2021) by Jim Freer

AR by Shaun Harrigan on behalf of the Authors (05 Apr 2021) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (08 Apr 2021) by Jim Freer

RR by Anonymous Referee #1 (14 May 2021)

Suggestions for revision or reasons for rejection

My apologies to the editor and the authors for the lateness of this review. Thanks very much to the authors for comprehensively addressing the first of my objections relating to CRPSS calculated with respect to persistence. It's clear from their explanation and additional figure that this was not an error, and I appreciate the detailed explanation and figure. (We can disagree on whether this result is 'intuitive' or not - in the responses the authors stated that the decline in accuracy of persistence is faster than GloFAS, which is what I (intuitively) assumed would happen, but it's clear this is not the case as skill falls with lead time. But this is basically irrelevant.) On re-reading the description of persistence skill (L234-235) there was one thing that was a little ambiguous to me - if a forecast is issued on Jan 2, where the first forecast value is for Jan 3, is the persistence forecast taken from Jan 1 or Jan 2? (I had assumed Jan 2, but I wasn't sure from their description.) This may be worth clarifying.

Added to this is that the many strengths of the paper remain: GloFAS is a system of international significance and this update is in my view ultimately absolutely worthy of publication. The paper is very clear and exceptionally well presented, and remains a join to read.

However, the authors chose not to address my second major comment about reliability (and I note they have the support of the editor in this, so I am probably howling into the wind here). Essentially I disagree with both the authors and the editor that omitting an assessment of reliability is acceptable. To me there are two fundamental reasons to present ensemble predictions (and I'm not alone here, going back at least as far as Krzystofowicz 2001): 1) The predictions are, on average, more accurate than deterministic predictions and 2) the forecasts are more 'honest' because they give a representation of predictive uncertainty.

The authors have clearly and admirably addressed (1) with their analysis of CRPS. The second they have ignored. I note also that Emerton et al. (2018) presented a basic assessment of reliability in their assessment of GloFAS-seasonal - I don't understand why such an analysis could not be performed here. (Though as I stated previously - a summary statistic of PIT values would be more appropriate, though attributes diagrams would be ok.) For the addition of one figure and perhaps a paragraph or two, the authors could (and in my view should) have addressed this fundamental aspect of ensemble prediction. If the authors were concerned about having too many figures, in my view they could remove Fig 11, which is unlikely to be read in detail and (presumably) can be accessed through the GloFAS portal.

References
Emerton, R, Zsoter, E, Arnal, L, Cloke, HL, Muraro, D, Prudhomme, C, Stephens, EM, Salamon, P, Pappenberger, F (2018) Developing a global operational seasonal hydro-meteorological forecasting system: GloFAS-Seasonal v1.0. Geosci. Model Dev. 11, 3327-3346.

Krzysztofowicz, R (2001) The case for probabilistic forecasting in hydrology. Journal of Hydrology 249, 2-9.

Hide

RR by Anonymous Referee #4 (28 Oct 2021)

Suggestions for revision or reasons for rejection

This is a review of "Daily ensemble river discharge reforecasts and real-time forecasts from the operational Global Flood Awareness System". I was asked to review this paper following a first round of revisions and have had the opportunity to evaluate the author responses to previous reviewers' comments. I think the authors have provided a thorough and convincing revision to these comments in the submitted manuscript.

As for my own personal review: I have read the paper and I think that this paper should be published promptly. This advancement seems to me as a fantastic contribution to the hydrological sciences community. I applaud the endeavor and think that HESS is a suitable journal for this type of contribution. I think the science (GloFAS model evaluation methods, figures and description of available forecast datasets) is sound and the end-product will be useful to many researchers. The structure is clear, concise and the text very well written.

I have but a few comments that I think the authors can easily address and which can be handled at the editorial board level.

1- Line 41: Australian --> Australia
2- Lines 56-57: Here the word "global/globe" is used 3 times in the span of 21 words. I suggest varying a bit as to not distract the reader.
3- Line 91: Reference to a future date that is actually in the past. Actually, most reforecast dates, IFS model cycles, versions, etc. need to be updated. I acknowledge that this is due to the review process, however I just want to highlight that this should be updated prior to publication.
4- General question: GloFAS-ERA5 uses ERA5 since it is updated near-real-time. However, ERA5-Land will soon also be near-real-time and will have the same native resolution as GloFAS (although the authors are better placed than I am to talk about this!). I wonder if there is any plan to update GloFAS-ERA5 to GloFAS-ERA5Land once it becomes near-real-time? If so, perhaps add a sentence on this in the concluding remarks or discussion?

Congratulations once again on the, to me, excellent paper.

Hide

RR by Anonymous Referee #5 (01 Dec 2021)

Suggestions for revision or reasons for rejection

--- Summary

The manuscript by Harrigan et al. was in its second round of reviews when I first reviewed it. The paper is of high quality and will be undeniably valuable to future users of the openly available GloFAS hindcasts. My review takes into account the replies of the authors to the previous reviews, replies which I found thorough and well supported. Nevertheless, after reading the manuscript, I still had questions similar to some of the comments previously raised, and therefore think some of these well justified choices deserve to be mentioned, even briefly, in the manuscript. Hereafter, I list some recommendations for improving explanations, some minor points, as well as a concern about Figure 5.

--- General comments

The authors have replied to comments made by previous reviewers on the subject, but I believe there should be in the paper an explanation for why the evaluation of a flood awareness system is not based on its capacity to forecast floods.

I wish the authors would better explain their choice of changing benchmark depending on the lead time, the reasoning behind the choice of the 0.5 threshold, and the expected influence on the results presented in Sections 4.2.2 and 4.2.3. For instance, why not choose the toughest benchmark to beat, designing for each catchment a benchmark based on both persistence and climatology that would resemble a locally optimized system switching from persistence to climatology when the CRPS of persistence and climatology cross?
I also have some concerns about the maps and some statements presented in Section 4.2.2. More specifically, to which extent does your choice of benchmark influence these spatial patterns? You chose to select the benchmark based on the lead time. Therefore, when looking at spatial skill patterns for lead times shorter than 15 days, won't the highest skills tend to be found in catchments where persistence is not adequate, i.e. catchments where seasonality dominates (hence the low skill in polar latitudes), rather than in places where GloFAS hindcasts are indeed of good quality? Reversely, at lead times longer than 15 days, the highest skills should be found in catchments where serial correlation dominates.
You mention L.294-295 that “The regions of highest skill are similar to those for short- and medium-range”, but this is not obvious when comparing areas without skill in Figure 6d and Figure 7a and it seems to me that what we observe are artifacts from this strict split in benchmark at 15 days lead. For instance, parts of Russia go from negative skills at lead time 10 to positive skills at lead time 15 days. This is counter-intuitive, and I think a comment from the authors would be necessary.
Finally and in line with these comments, I question the use of “should be” L.432-433 in the conclusion: “The analysis shows that on average across the world forecasts should be benchmarked against persistence for lead times up to about two weeks and against climatology for longer lead times.” It currently sounds as a recommendation, but it seems that this analysis should be performed and fine-tuned for use at non-global scale.

In several of your replies to R2, you mentioned that a novelty from this work was that the reforecasts, along with other data types, are open. Opportunities for further research based on this data are well mentioned, but I expected a deeper discussion on the incentives for opening up and on encountered challenges. I believe your experience may be valuable for other data providers choosing to switch to open access, and highlighting overcome challenges would only highlight the value of this work.

There seems to be an inconsistency between Figures 5a and 5b. It is unclear why the intersection between the median CRPS of persistence and the median CRPS of climatology (~18 days) does not correspond to the intersection of the median CRPSS (~14 days). I could not find any reasonable reason why this would happen.

--- Minor comments

L.14 Can we really say that global-scale hydrological forecast systems are widely used ?

L. 80-81 ‘reforecasts’ (as ?) consistent as possible

L.92 ‘of with’ seems strange

L.140-143 The parenthesis count is off.

L.165 The second incentive for using hindcasts is not as universal as incentives 1) and 3). In fact, the reader only links IFS updates and influence on the hindcasts in the next paragraph. I suggest moving incentive 2) later in the text, and use this information not as a common incentive to using hindcasts, but rather as additional information on the choice of 2019 as reference year for the system versions.

Figure 3 For the sake of showing a self-explanatory figure, I suggest explaining « ens=11 » in the caption or explicitly writing « 11 members ».

Figure 4 KGESS should be briefly explained when presenting this figure. The authors offered a short and clear explanation in their reply to previous comments which could very well fit in the paper and ease understanding.

L.271-272 “At day 15…” This sentence is unclear.

L.276 “The aid the readers…” seems incorrect. Did you mean “To aid the readers…”?

L.287 “across at” sounds strange.

Figure 8a Please add an explanation in the caption for the red and blue-shaded areas.

L. 352 Remove “be”

L.434 “… results show...”

Hide

ED: Publish subject to revisions (further review by editor and referees) (14 Jan 2022) by Jim Freer

AR by Shaun Harrigan on behalf of the Authors (26 Mar 2022) Author's response

ED: Publish as is (01 Jul 2022) by Jim Freer

AR by Shaun Harrigan on behalf of the Authors (19 Aug 2022)

Short summary

Real-time river discharge forecasts and reforecasts from the Global Flood Awareness System (GloFAS) have been made publicly available, together with an evaluation of forecast skill at the global scale. Results show that GloFAS is skillful in over 93 % of catchments in the short (1–3 d) and medium range (5–15 d) and skillful in over 80 % of catchments in the extended lead time (16–30 d). Skill is summarised in a new layer on the GloFAS Web Map Viewer to aid decision-making.