From skill to value: isolating the influence of end user behavior on seasonal forecast assessment

Giuliani, Matteo; Crochemore, Louise; Pechlivanidis, Ilias; Castelletti, Andrea

doi:https://doi.org/10.5194/hess-24-5891-2020

Articles | Volume 24, issue 12

https://doi.org/10.5194/hess-24-5891-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-24-5891-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 24, issue 12

Research article

|

11 Dec 2020

Research article |

| 11 Dec 2020

From skill to value: isolating the influence of end user behavior on seasonal forecast assessment

Matteo Giuliani, Louise Crochemore, Ilias Pechlivanidis, and Andrea Castelletti

Download

Final revised paper (published on 11 Dec 2020)
Preprint (discussion started on 08 Jan 2020)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review report hess-2019-659', Anonymous Referee #1, 07 Feb 2020
- AC1: 'Reply to Anonymous Referee #1', Matteo Giuliani, 31 Mar 2020
RC2: 'Review of hess-2019-659', Anonymous Referee #2, 19 Feb 2020
- AC2: 'Reply to Anonymous Referee #2', Matteo Giuliani, 31 Mar 2020
EC1: 'Final Editor's comment', Dimitri Solomatine, 20 Apr 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Publish subject to revisions (further review by editor and referees) (20 Apr 2020) by Dimitri Solomatine

AR by Matteo Giuliani on behalf of the Authors (18 May 2020) Manuscript

ED: Referee Nomination & Report Request started (08 Jun 2020) by Dimitri Solomatine

RR by Anonymous Referee #1 (12 Jun 2020)

RR by Anonymous Referee #2 (10 Aug 2020)

Suggestions for revision or reasons for rejection

Rather than rework their study in response to comments raised, the authors have opted to provide a full rebuttal. I’m afraid I remain unconvinced of the key contribution having carefully considered the responses.

In brief:

- The authors retain their claim that the paper provides a novel framework, based on the argument that the input data are state-of-the-art, reservoir model is accurate, and so on. I don’t doubt any of these claims. However, for the framework to be novel it must involve more than one-way coupling of forecasts -> reservoir model -> ag profit model. This is just linking output of one model to another, and not a novel framework.
- I agree with the authors that the value Euro490million is a valuable insight, particularly for those with an interest in the region. But it’s very case specific and not a broad enough contribution to knowledge for a scientific paper.
- Modeling risk aversion of decision makers through use of different forecast percentiles is still poorly justified in my view.
- The authors continue to press the claim of an exponential relationship given the very limited nature of the inputs and the fact that full simulation period results are combined with 2005-only results in this graphic. The authors acknowledge that this is problematic, conceding that more points would make the analysis more statistically sound. The authors' response is that generating more points is too computationally intensive. If this is the case, then the associated conclusion needs to be removed or properly caveated. The abstract reads “…our results show an exponential skill to value relation where large gains in forecast skills are necessary to generate moderate gains in end-user profit”. This is a powerful statement and must be backed up with rigor.

(Also, I don’t think “skill” is actually defined qualitatively or mathematically anywhere in the paper)

Hide

ED: Publish subject to revisions (further review by editor and referees) (11 Aug 2020) by Dimitri Solomatine

AR by Matteo Giuliani on behalf of the Authors (21 Sep 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (14 Oct 2020) by Dimitri Solomatine

RR by Anonymous Referee #2 (18 Oct 2020)

ED: Publish as is (04 Nov 2020) by Dimitri Solomatine

AR by Matteo Giuliani on behalf of the Authors (05 Nov 2020)

Short summary

This paper aims at quantifying the value of hydroclimatic forecasts in terms of potential economic benefit to end users in the Lake Como basin (Italy), which allows the inference of a relation between gains in forecast skill and gains in end user profit. We also explore the sensitivity of this benefit to both the forecast system setup and end user behavioral factors, showing that the estimated forecast value is potentially undermined by different levels of end user risk aversion.