Preprints
https://doi.org/10.5194/hess-2022-65
https://doi.org/10.5194/hess-2022-65
 
21 Mar 2022
21 Mar 2022
Status: a revised version of this preprint is currently under review for the journal HESS.

Flexible forecast value metric suitable for a wide range of decisions: application using probabilistic subseasonal streamflow forecasts

Richard Laugesen1,2, Mark Thyer1, David McInerney1, and Dmitri Kavetski1 Richard Laugesen et al.
  • 1School of Civil, Environmental and Mining Engineering, University of Adelaide, SA, Australia
  • 2Bureau of Meteorology, Canberra, ACT, Australia

Abstract. Forecasts have the potential to improve decision-making but have not been widely evaluated because current forecast value methods have critical limitations. The ubiquitous Relative Economic Value (REV) metric is limited to binary decisions, cost-loss economic model, and risk neutral decision-makers. Expected Utility Theory can flexibly model more real-world decisions, but its application in forecasting has been limited and the findings are difficult to compare with those from REV. A new metric, Relative Utility Value (RUV), is developed using Expected Utility Theory. RUV has the same interpretation as REV which enables a systematic comparison of results, but RUV is more flexible and able to handle a wider range of real-world decisions because all aspects of the decision-context are user-defined. In addition, when specific assumptions are imposed it is shown that REV and RUV are equivalent. We demonstrate the key differences and similarities between the methods with a case study using probabilistic subseasonal streamflow forecasts in a catchment in the southern Murray-Darling Basin of Australia. The ensemble forecasts were more valuable than a reference climatology for all lead-times (max 30 days), decision types (binary, multi-categorical, and continuous-flow), and levels of risk aversion for most decision-makers. Beyond the second week however, decision-makers who were highly exposed to damages should use the reference climatology for the binary decision, and forecasts for the multi-categorical and continuous-flow decision. Risk aversion impact was governed by the relationship between the decision thresholds and the damage function, leading to a mixed impact across the different decision-types. The generality of RUV makes it applicable to any domain where forecast information is used for making decisions, and the flexibility enables forecast assessment tailored to specific decisions and decision-makers. It complements forecast verification and enables assessment of forecast systems through the lens of customer impact.

Richard Laugesen et al.

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on hess-2022-65', Anonymous Referee #1, 27 May 2022
    • AC1: 'Reply on RC1', Richard Laugesen, 30 Aug 2022
  • RC2: 'Comment on hess-2022-65', Anonymous Referee #2, 13 Jul 2022
    • AC2: 'Reply on RC2', Richard Laugesen, 30 Aug 2022

Richard Laugesen et al.

Richard Laugesen et al.

Viewed

Total article views: 816 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
654 144 18 816 41 8 7
  • HTML: 654
  • PDF: 144
  • XML: 18
  • Total: 816
  • Supplement: 41
  • BibTeX: 8
  • EndNote: 7
Views and downloads (calculated since 21 Mar 2022)
Cumulative views and downloads (calculated since 21 Mar 2022)

Viewed (geographical distribution)

Total article views: 744 (including HTML, PDF, and XML) Thereof 744 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 02 Dec 2022
Download
Short summary
Forecasts may be valuable for user decisions but current practice to quantify it has critical limitations. This study presents a new metric that can be tailored to specific decisions and decision-makers, and shows that streamflow forecasts out to 30 days provide high value for almost all users, but not always. This study paves the way for agencies to tailor the evaluation of their services to customer decisions, and researchers to study model improvements through the lens of user impact.