A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context

Berthet, Lionel; Bourgin, François; Perrin, Charles; Viatgé, Julie; Marty, Renaud; Piotte, Olivier

doi:https://doi.org/10.5194/hess-24-2017-2020

Articles | Volume 24, issue 4

https://doi.org/10.5194/hess-24-2017-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-24-2017-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 24, issue 4

Research article

|

23 Apr 2020

Research article |

| 23 Apr 2020

A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context

Lionel Berthet, François Bourgin, Charles Perrin, Julie Viatgé, Renaud Marty, and Olivier Piotte

Download

Final revised paper (published on 23 Apr 2020)
Supplement to the final revised paper
Preprint (discussion started on 07 May 2019)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review for HESS-2019-181', Anonymous Referee #1, 17 Jun 2019
- AC1: 'Reply to both referees', Lionel Berthet, 10 Aug 2019
RC2: 'Review of A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context', Kolbjorn Engeland, 21 Jul 2019
- AC2: 'Reply to both referees', Lionel Berthet, 10 Aug 2019
EC1: 'Editor's comment, before revision', Dimitri Solomatine, 17 Aug 2019

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Publish subject to revisions (further review by editor and referees) (17 Aug 2019) by Dimitri Solomatine

AR by Lionel Berthet on behalf of the Authors (26 Sep 2019) Author's response Manuscript

ED: Referee Nomination & Report Request started (04 Oct 2019) by Dimitri Solomatine

RR by Kolbjorn Engeland (29 Oct 2019)

Suggestions for revision or reasons for rejection

I find that the paper has improved and can be published after some minor clarifications. The paper presents a topic of high interest, the presentation is now consistent, it will be easy for other researcher to repeat a similar experiment.

Page 9, Line 24-25
Please add reference in this sentence:
“In previous work, we used the log transformation because it ensures that no negative values are obtained when estimating the predictive uncertainty for low ﬂows.”

Page 10 Line 1-10
It is not necessary to provide many details for a method you did not use. Either delete this part or summarize the content in maximum two sentences.

Page 12 and Figure 4.
For the event selection you write that “the event was kept […] if the peak value was higher than 50% of the highest discharge value of the time series.

In Figure 4 that illustrates the event selection, it seems like the highest peak is around 110 m3/s, whereas some of the events with the lowest flood peaks are much smaller than 55 m3/s that is 50% of the highest peak. Could you explain?

Page 12 Line 10-15. The calibration and verification procedure could be better explained, see suggestion below.
“We used a two-step procedure, as illustrated in Fig. 5. The ﬁrst step was to calibrate the parametric transformations. For each transformation and for each parameter set, the empirical residuals were computed over D1 and D2sup where the EHUP was trained on the highest flow group of D1 (see section 2.1.3) and the calibration criterion was computed on D2sup. Indeed, the data transformations have almost no impact on the uncertainty estimation by EHUP on events of the same magnitude as those of the training subset. Therefore, the calibration subset has to encompass events of a larger magnitude (D2sup). The parameter set obtaining the best criterion value was selected. In the second step, the EHUP was trained on the highest flow group of D1, D2inf and D2sup combined and using the parameter set obtained during the calibration step. Then, the predictive uncertainty distribution was evaluated on the control data set D3. “

Figure 6, 10, 12, 13, 14,15: Maybe add one sentence in the figure captions what the dashed line represents.

Why have you chosen the alpha-index to have a optimal value at 1 whereas the other scores have an optimal value at 100% ?

Page 27, : Line 29: For clarity you could write: We used the Shapiro-Francia test where the zero hypothesis is that the data are normally distributed.

Hide

RR by Anonymous Referee #1 (21 Nov 2019)

ED: Publish subject to revisions (further review by editor and referees) (03 Dec 2019) by Dimitri Solomatine

AR by Lionel Berthet on behalf of the Authors (13 Jan 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (22 Jan 2020) by Dimitri Solomatine

RR by Kolbjorn Engeland (26 Jan 2020)

RR by Anonymous Referee #1 (24 Feb 2020)

ED: Publish subject to technical corrections (15 Mar 2020) by Dimitri Solomatine

AR by Lionel Berthet on behalf of the Authors (21 Mar 2020) Author's response Manuscript

Short summary

An increasing number of flood forecasting services assess and communicate the uncertainty associated with their forecasts. We present a crash-testing framework that evaluates the quality of hydrological forecasts in an extrapolation context. Overall, the results highlight the challenge of uncertainty quantification when forecasting high flows. They show a significant drop in reliability when forecasting high flows and considerable variability among catchments and across lead times.