This is my second review of the article entitled "Moving the cost-loss ratio: Economic assessment of streamflow forecasts for a risk-averse decision maker" submitted to Hydrology and Earth System Sciences by Matte et al.
This article offers a 'new' approach for the economic assessment of flood forecasts, which takes into account the risk aversion of many end users, which is not the case in the "classical" cost-loss ratio methodology ('new' in the hydrological field). Then, this paper addresses an economic issue which would interest many hydrologists, in the operational services as well as academics. The theoretical framework for the assessment is experimented in a practical case study: the 2004 floods of the Montmorency River (Québec, Canada).
First, I have to thank the authors for their very clear, precise and honest answers to my first comments (in the discussion). They helped me to understand much better their approach. I also congratulate them for the very serious work which led to this new version of the manuscript. Many - if not all - comments of the 2 reviewers have been taken into account. It is much clearer (in my opinion) and which is now almost ready for publication in HESS. Thanks to the added explanations and references, it is now much more adapted for non economist readers (such as many hydrologists). The results and discussion sections are particularly clear. However, one main difficulty and a few minor ones remain (see below). I therefore recommend a minor revision to allow the authors to add some final improvements to their paper (in my opinion, it does not require another revision round any more).
MAIN COMMENTS
Before discussing the main issue ("weakness") remaining in the article, I would like to continue the discussion about the cost-loss ratio approach and the notion of risk aversion. One point I still not understand is the possibility of a link between 2 ideas :
a) risk aversion: how much an individual is eager to pay to be insured against an event ;
b) the optimal ratio of false alarms and missed events for an individual, that is the number of false alarms (which cost a little amount of money c1) in order to avoid more than 1 missed event (which costs a large amount of money c2); this question can be treated by the cost-loss ratio.
Since both notions are related to the amount of money an individual is eager to pay to avoid the consequences of any (respectively: more than 1) event, it is not clear for me whether we can make a link between these 2 approaches rather than opposing them.
As pointed out by the authors in the discussion and in their revised manuscript, the results of the experiment on the case study are significant only if the data is representative of the catchment behaviour (see the notion of expectation, page 13). The case study here is driven over 4 years (2011-2014), thus over a (very) limited number of (significant) events. The sufficiency of data (to infer any conclusion) should be explicitly treated. It may be useful to:
a) display the observations and the forecasts (at least in the supplementary materials);
b) sum up all the events (e.g. discharge over a chosen threshold), and to pay a particular attention to missed events and successful forecasts: I expect that their "ratio" would strongly influence the results of this study.
DETAILED COMMENTS
Page 2, lines 5 - 6: if available, some reference would be useful.
Page 2, line 14 ("the latter gain in importance"): check grammar ("gains"?)
Page 2, line 15 ("However, AS there exist many sources of uncertainty in hydrological processes, there also exist many means [...]"): I did not get the logical implication between these 2 ideas.
Page 2 line 15 ("there also exists many means"): check grammar ("exist"?)
Page 2, line 22: since the "value" word is in italic 2 lines further, I suggest to use italic for the "quality" word as well.
Page 2, line 25: isn't a point sign missing before the words "In particular"?
Page 2, line 26 ("In particular, the usefulness of a forecast is inherently linked to the decision maker's ability to adapt their behaviour to the information provided"): I totally agree with this sentence but why is this important in the introduction? (why to state this issue here?) In a more general way, it is the ability to use the information which allows a forecast to be useful.
Page 4, line 29 ("the decision maker may only distinguish between a finite set of implied damages"). I am not sure of what the reader should understand here; is this a practical observation or a more theoretical affirmation?
Page 5, lines 11-12: some references (such as those given in appendix B) could usefully be provided here.
Page 6, line 15: since the month of the floods of 2012 and 2014 are given below, to precise in which month the 1964 flood occurred could be useful and consistent.
Page 6, lines 21-22 ("The greatest concern of public authorities occurs when people refuse to evacuate [...]"): this is unfortunately a too common behaviour. However, why is this information useful here to demonstrate the pertinence of the proposed economic assessment?
Page 6, lines 25 and following: very little is said about the spatial resolution of the model (how many RHHU? ...). It is not clear how well it is adapted to the data inputs and to the catchment (in particular, to state that it is a physics-based model).
Page 7, lines 3-5. The model description has been well improved (compared to the first submission). However, I still don't understand why details such as those about the vertical water budget scheme (BV3C) are relevant for this publication.
Page 8, line 19 ("In a study involving 20 catchments in Quebec"). I acknowledge the fact that the interested reader can easily find Thiboult et al. (2016). However, it could be worth providing a few details about these catchments (can we compare them to the Montmorency River catchment used in this case study?).
Page 8, line 20 ("the uncertainty for initial conditions dominate the other sources of uncertainty for short term (1-day to 3-day ahead)"): 3 days seem to be a very long period compared to the response time (12h, according to page 6, line 6). Is this correct?
Page 9. The description of the "rudimentary" data assimilation scheme is much clearer in this new version. However, I am still not convinced that it can be called an Ensemble Kalman filter (there is no sequential approach here). At least, the absence of a sequential scheme could usefully be pointed out.
Page 9 ("The inclusion of additive perturbations for precipitation is due to the fact that strong under-captation is suspected for this catchment."): I don't understand this. Do the authors mean "additive positive perturbations"?
Page 10, lines 1-3: the 'm' subscript is used for 2 different items: the ensemble members and the damage categories. This could infer some confusion.
Page 13, line 22 ("On the other hand, there can also be various sources of non-stationarity [...]"): this argument is rather specious. This may indeed occur but there are means to detect non-stationarity. The conclusion (lines 23-24) can not (should not?) be inferred from this argument. The representation of the expectation of the utility by its average value could be better discussed (see the main comments).
Page 17, line 28 ("We find that risk-averse end-users mainly consider the less favourable scenarios"): is this really a finding of this study? If so, it should be stated "with/in this model, we find [...]".
Page 18, lines 23-24 ("in this paper, we did not address the issue of potential cognitive biases and training issues for end-users"): I agree and thank the authors to point out these issues. In addition, it could be worth clearly stating that risk aversion is not a cognitive bias (see the answers of the authors in the discussion, page C10), e.g. page 10 after lines 21-22.
Page 27, Tab. 2: I don't understand the horizontal line under "No limit for a 1-day forecast". Should other lines (above and under "No limit for a 5-day forecast") be added?
Page 28, Fig. 1: this figure is a "schematic" representation of the CARA utility function for A > 0. Why not showing a real CARA utility function for A > 0? It could help the reader keeping in mind that the CARA function values are all negatives in such a case?
Page 28, Fig. 1 caption: is this equation coherent with the equation 2 (page 5)? Furthermore, I am also uncomfortable with this equation. Since C is the difference between 2 utility values, it is therefore not a money amount (see page 14, line 7: "the actual value of the decision maker's utility has no interpretation). Then, if the equation in the caption is correct, why and how could C be the amount that the individual would be willing to pay up to? |
- Printer-friendly version
- Supplement
