Estimating the benefits of single value and probability forecasting for flood warning

Flood risk can be reduced by means of flood forecasting, warning and response systems (FFWRS). These systems include a forecasting sub-system which is imperfect, meaning that inherent uncertainties in hydrological forecasts may result in false alarms and missed events. This forecasting uncertainty decreases the potential reduction of flood risk, but is seldom accounted for in estimates of the benefits of FFWRSs. In the present paper, a method to estimate the benefits of (imperfect) FFWRSs in reducing flood risk is presented. The method is based on a hydro-economic model of expected annual damage (EAD) due to flooding, combined with the concept of Relative Economic Value (REV). The estimated benefits include not only the reduction of flood losses due to a warning response, but also consider the costs of the warning response itself, as well as the costs associated with forecasting uncertainty. The method allows for estimation of the benefits of FFWRSs that use either deterministic or probabilistic forecasts. Through application to a case study, it is shown that FFWRSs using a probabilistic forecast have the potential to realise higher benefits at all lead-times. However, it is also shown that provision of warning at increasing leadtime does not necessarily lead to an increasing reduction of flood risk, but rather that an optimal lead-time at which warnings are provided can be established as a function of forecast uncertainty and the cost-loss ratio of the user receiving and responding to the warning. Correspondence to: J. S. Verkade (jan.verkade@deltares.nl)


Introduction
Floods are an act of God but flood damage is an act of Man (White, 1942).For long though, flood management has primarily focused on managing flood hazards, e.g. on reducing the frequency of flooding, flood extent, depth and duration and flow velocities.Recent years have seen an increased emphasis on the management of flood risk, where risk is defined as the combination of the probability of occurrence of a flood event, and its consequences in terms of casualties and economic damage (Merz et al., 2010).This shift from flood hazard management to flood risk management has led to an increased emphasis on non-structural measures including, for example, spatial planning, raising flood awareness, flood proofing and the use of flood forecasting, warning and response systems (FFWRSs).
Of these flood risk management measures, flood warning is regarded as being one of the most effective (UNISDR, 2004).Considerable attention has been given to the effectiveness of these systems.These studies generally focus on estimating flood losses, the potential reduction of these losses through warning response and the relationship between flood warning lead-time and loss reduction (e.g.Parker, 1991;Carsell et al., 2004;Parker et al., 2008;Molinari and Handmer, 2011).
Flood forecasts, which form an essential element in the flood forecasting, warning and response process are, unfortunately, affected by inherent uncertainties.These pertain to the forecasting model structure, parameter values and initial conditions, to meteorological forcing (especially when this forcing is forecast rather than observed), and to measurements and interpolations of these measurements as for example in deriving catchment average rainfall.This forecasting uncertainty can be explicitly accounted for if the forecasting sub-system of a FFWRS produces an estimate of predictive uncertainty as in the case of probabilistic forecasting.
Irrespective of the nature of the forecasting system, this forecasting uncertainty can lead to "wrong" decisions: floods that occur may not have been predicted in time, or floods that are predicted may not occur.The costs associated with this forecasting uncertainty can be considerable.An analysis of the role of benefits of FFWRSs should therefore also include these costs, consisting of an opportunity cost in the case of a flood that was not predicted, and the cost of unnecessary warning response in the case of a false alarm.
Flood risk can be defined as the expected value of flood related damage and costs.Floods are random events and therefore flood damage is a random event.Although the exact amount of damage in any given year cannot be predicted, the expected annual value of flood damage can be determined if the probability distribution of flood damage, or damagefrequency curve is known.This expected annual damage is a measure of flood risk.Flood risk may be estimated using a hydro-economic Expected Annual Damage (EAD) model (USACE, 1994;Dingman, 2002;Loucks et al., 2005), which uses three basic relationships to establish the probability distribution of flood damage: the flood frequency curve, the rating curve and the stage-damage curve.
To evaluate the benefit of measures taken to reduce flood risk, the cost of these measures should be taken into account.In the case of flood warning systems, such an analysis should include the expected reduction of flood losses due to the provision of warning and subsequent response, as well as the costs of operating such systems and the costs associated with uncertainty.Whilst the first two of these can be readily incorporated in analysing the benefit of flood warning, the latter is less straightforward.
In meteorological applications, Relative Economic Value (e.g.Murphy, 1985;Zhu et al., 2002) is often used to establish the value of forecasting systems relative to two benchmark situations.These are the situations in which no warning system is present, and the situation in which a perfect warning system is present.In the latter, forecasting uncertainty is absent and hence no "wrong" decisions are ever made.
To the best of our knowledge, no flood risk analyses have been published that include the damage mitigating effects of flood warning, the costs of the warning system, and the costs associated with forecasting uncertainty.In the present paper, a method is proposed that can be used to estimate flood risk in the presence of an imperfect FFWRS.The method consists of combining the hydro-economic EAD model with the theory of Relative Economic Value.This combines expected annual damage, loss reduction, cost of warning response and the costs associated with forecasting uncertainty

Mitigation time
Lead time
into an estimate of the benefit of flood forecasting and warning in reducing flood risk.
This method allows for the comparison of the effect of flood risk management measures of different nature.For example, the flood risk reduction attained by the implementation of a flood warning system can be compared with that attained by the raising of levees, installation of flood retention areas or increasing flow conveyance.Additionally, the method allows for an intercomparison of FFWRSs.For example, the benefit of systems based on deterministic forecasting can be compared with those that are based on probabilistic forecasting.This allows explicitly estimating the benefit of probabilistic forecasting in terms of flood risk reduction, which so far has only been described in terms of their potential for improved decision making in flood event management (e.g.Krzysztofowicz, 2001;Todini, 2004).
In the next section, the proposed method is explained in detail.In Sect.3, results of a case study are presented where the method is demonstrated by application to a small basin.The results are discussed in Sect. 4. Finally, a summary and brief conclusions are presented in Sect. 5.

Flood forecasting, warning and response systems
A properly working flood forecasting, warning and response system (FFWRS) gives property owners, floodplain residents and responsible authorities time to respond to a flood threat before flooding occurs.FFWRSs usually consist of a number of sub-systems (Fig. 1).The forecasting sub-system produces forecasts of hydrological variables such as water levels or flow rates, either as a deterministic single value forecast or as a probability distribution.Based on these forecasts, a decision is taken whether or not to initiate warning response.The warning-response sub-system then consists of warning procedures and subsequent mitigation action that can be taken to reduce flood losses.
Although in actual operational forecasting the decision to warn will be taken by the forecaster using guidance from the forecasting sub-system, in the present paper it is assumed Hydrol.Earth Syst.Sci., 15, 3751-3765, 2011 www.hydrol-earth-syst-sci.net/15/3751/2011/ that decisions are based on forecasts only.Depending on the nature of the forecasting sub-system, the decision subsystem is deterministic or probabilistic.In the case of deterministic forecasts, it is assumed that forecast water levels that are higher than the flooding threshold will automatically initiate a warning response.Essentially, this decision is then taken implicitly by the forecaster.If the forecasting system provides explicit estimates of predictive uncertainty, the decision will have to be based on a probabilistic decision rule.
If the probability of forecast water levels exceeding the flooding threshold is higher than a probability threshold, a warning response will be initiated.This allows users to choose an optimal threshold (in terms of probability threshold) at which mitigating action is initated (Krzysztofowicz, 2001), but it is again assumed here that forecast probabilities higher than the selected probability threshold will automatically initiate a response.
The warning-response sub-system pertains to the damagemitigating actions that can be taken after a flood warning has been issued.During the time between a flood warning and the arrival of flood waters -the mitigation time -floodplain residents can move themselves and/or their property out of reach of the pending flood.Increasing the available mitigation time intuitively allows for increased loss reduction, and therefore this mitigation time should be maximised (but note that with increasing mitigation time, response costs may increase as well).Forecasting lead-time and mitigation time are different due to the time needed to produce and disseminate a forecast and to take a decision whether or not to initiate a warning response (Fig. 1) (Carsell et al., 2004).However, in the context of the present paper the time taken in the decision sub-system is negligible and lead-time and mitigation time are used synonymously.
Maximum potential reduction of flood damage by flood warning response is rarely attained as it is unlikely that all floodplain residents will be notified in time, nor that all residents will heed the warnings and act effectively.To account for this, Parker (1991) and Green and Herschy (1994) defined the actual flood damage avoided L a [GBP] as a product of the maximum potential flood damage avoided with a fully effective system (L p [GBP]), the probability that a forecast is made in time (R [−]), the fraction of residents available to respond to a warning (P a [−]), the fraction of residents who will respond to a warning (P r [−]) and the fraction of households who respond effectively (P e [−]).Together, these probabilities and dimensionless factors, each ranging from 0 to 1, represent the effectiveness of the response: L a = L p × R × P a × P r × P e .In the UK, the Department for Environment, Food and Rural Affairs (DEFRA) indicated the values for the factors and probabilities (R, P a , P r , P e ) the Environment Agency seeks to achieve (DEFRA, 2004).These would result in L a = 0.5 × L p , which is the value used in the present paper.

Expected annual flood damage
Flooding is a random process and therefore flood damage is a random process.The expected value of annual direct, tangible flood damage can be estimated from the probability distribution of flood damage: where P is the annual probability of exceedence of a certain flood level and D(P ) is the direct, tangible flood damage caused by that flood event (e.g.USACE, 1994;Carsell et al., 2004;de Bruijn, 2005;Loucks et al., 2005).To determine the probability distribution of flood damage, the hydro-economic EAD model (USACE, 1994;Davis et al., 2008;Dingman, 2002;Loucks et al., 2005) links the flood frequency distribution through flood stages to flood damage.The model can best be explained graphically (Fig. 2).The starting point of the analysis is the probability distribution of flow rates (or flood frequency curve, bottom left panel).A rating curve (top left panel) links flow rates to flood stages.Stages higher than the flooding threshold will cause damage, described by the stage -damage curve shown in the top right quadrant.By linking the probability of each flood discharge to the stage in the river to the damage occurring, the probability distribution of flood damage D(P ) can be established (bottom right panel).The expected annual flood damage can then be easily established as the area enveloped by the probability-damage curve (Eq.1).The effect of flood risk management measures can easily be shown in the graphical model.Measures that reduce flood frequencies push the flood frequency curve (bottom left) towards the origin.Measures aimed at a reduction of flood stage, e.g. by river bed deepening or widening, change the rating curve (top left panel).The reduction of flood damage, either by structural or by non-structural measures, reduce damage associated with flood stages (top right panel).Ultimately, measures that are effective in reducing flood risk will move the probability-damage curve towards the origin (Dingman, 2002), thus reducing the expected annual damage.
Figure 2 shows an example of the effect of a flood risk management measure.Here, a measure was implemented that reduces flood damage.Such a measure could be, for example, flood-proofing private properties.The measure does not affect either the probability of flooding or the rating curve, but does change the stage -damage relationship, with a reduced damage expected at the same stage.This results in a probability -damage relationship that lies closer to the origin, with the expected annual damage being reduced.

Cost of flood warning response and cost-loss ratio
Flood forecasting, warning and response systems come at a cost, consisting of initial costs for setting up the system, fixed costs for operation and maintenance, and variable event costs for flood warning response; the latter are incurred every time a warning is issued.The fixed costs can be included in the EAD analysis by adding these to flood damage, and shifting the stage-damage curve to the right.Strictly speaking, the term "damage" is then incorrect as it also includes the cost of measures.In this paper, it is assumed for simplicity that the fixed costs are included in the event costs.Additionally, the event costs are considered independent of the height of the flood stage.This is considered reasonable as the cost of response is incurred based on a forecast (probability) threshold being exceeded, and therefore independent of the actual height with which the threshold is exceeded.The cost-loss ratio r in Eq. ( 2) can be used to express the costs of warning response C as a fraction of the avoidable losses L a .It is clear that where r > 1 there is no benefit in flood warning response, whilst for a very low C the ratio r approaches 0. (2)

Relative economic value
If a decision to initiate warning and response procedures is solely based on an imperfect forecast, forecasting uncertainty may lead to false alarms and missed events.Both false alarms and missed events are instances of imperfect system performance and adversely impact the potential reduction in flood risk.Combining the hydro-economic EAD model with the theory of Relative Economic Value (e.g.Murphy, 1985;Zhu et al., 2002) offers a convenient way of incorporating the costs associated with forecasting uncertainty in estimates of expected annual damage.Using the hydro-economic EAD model, flood risk can be estimated for the No Warning and for the Perfect Warning cases.Zhu et al. (2002) define the Relative Economic Value (REV) as a dimensionless factor to scale between these estimates.The maximum value of 1 is assigned to the Perfect Warning case, while a warning system that has the same skill as the climatology (here meaning the long-term average frequency of flooding) is assigned 0. Given the low climatological frequency of flood threshold exceedance, this can be considered equivalent to the case with No Warning being present.The REV can be calculated based on the skill of the FFWRS.
The performance of a FFWRS can be captured in a twoby-two contingency table that shows forecast/observation pairs for dichotomous events (Wilks, 2006).In this case, the table shows in how many cases a flood warning was followed by a flood event (Table 1).A contingency table is based on a record of forecasts and events and should be made for every decision rule that is used.
In the absence of a FFWRS, a user's flood losses will be determined by the climatological frequency of flooding and consist of unmitigated losses, which is the sum of the losses avoided through warning response L a , and the losses that cannot be avoided L u for every flood event: (3) If a FFWRS is based on perfect forecasts, a flood event is always preceded by a warning and flood damage can always be reduced by mitigating action.False alarms and missed events do not occur.The expected damage then consists of the sum of cost for warning response and unavoidable losses for every flood event: The performance of a FFWRS based on imperfect forecasts can be assessed using a contingency table.Missed events result in unmitigated flood losses, which equal the sum of avoidable and unavoidable losses L a + L u .Loss mitigation through warning response can only be achieved at a cost C.In case of false warnings, these are the only costs incurred by a user.A user's expected costs and losses consist of those associated with hits, misses and false alarms: (5) The Relative Economic Value (V [−]) of an imperfect warning system is defined as the value relative to the benchmark cases of No Warning (V = 0) and Perfect Forecasts (V = 1): Note that REV can be less than 0 if the cost of false alarms is higher than the benefits attained by the warning system.
Substituting Eqs. ( 3), ( 4) and ( 5) in ( 6), subsequent division by L a and substitution of C/L a by r (Eq.2) yields: This derivation of relative economic value slightly differs from that of, for example, Zhu et al. (2002).The difference is in the expected expense in the absence of a warning system.Zhu et al. include an additional decision where, based on a minimisation of cost, a user may decide either to never, or to always take action.In the latter case, a single warningresponse action is assumed to have an impact that is unlimited in time, leading to an expected expense of C + oL u .Including this EAD nowarn = min[o(L a + L u ), C + oL u ] in the analysis would yield relative economic value as a function of min(o, r) which is discontinuous at r = o.In the present application, the climatological frequency of flooding o approaches 0 and most if not all users' cost-loss ratio r is greater than o.For that reason, the present derivation may be simplified.It may be noted that flood risk in the "always take action" option may be estimated by using the hydroeconomic EAD-model.

Optimal warning rule
It is assumed that a decision to issue a warning will only be taken if the expected value of the warning response is less than the expected value of not issuing a warning.This yields the optimal warning rule: P > C L a P > r with P the predicted probability of flooding.Only if a user applies the optimal warning rule to flood event decision making, will the benefits of probability forecasting be fully realised.

Combining expected annual damage with relative economic value
Flood risk in the No Warning and Perfect Forecasts cases can be calculated using the hydro-economic EAD-model.This equally yields EAD nowarn and EAD perfect respectively.To calculate EAD FFWRS , REV is subsequently used to scale between the flood risk of benchmark cases using Eq. ( 6): www.hydrol-earth-syst-sci.net/15/3751/2011/ Hydrol.Earth Syst.Sci., 15, 3751-3765, 2011 In words: the flood risk in case of a warning system being present equals the flood risk in the absence of such a system minus the avoidable risk, which is scaled by the warning system performance.A perfect system (where V = 1) brings the full benefits of a warning system (EAD FFWRS = EAD perfect ).A system that performs as well as acting on climatological information (V = 0) does not bring any additional benefits, and is equivalent to no warning system being present: EAD FFWRS = EAD nowarn .A system that brings benefits compared to the absence of a warning system (0 < V < 1) will result in an expected annual damage between that of the benchmark cases: EAD nowarn > EAD FFWRS > EAD perfect .If the warning system performance is worse than that in the No Warning case (V < 0), flood risk will increase to levels higher than that in the No Warning case: EAD FFWRS > EAD nowarn .In that case, there is no economic rationale for flood warning.
As the potential for loss mitigation increases with increasing lead-time provided by the warning system, flood risk in the presence of a FFWRS is different for different leadtimes: EAD FFWRS = f (n) (where n is lead-time).Additionally, Eq. ( 7) shows that relative economic value is expressed as a function of the users' cost-loss ratios: V = f (r).Explicitly including these dependencies in Eq. ( 9) gives: The assumption that was made here is that flood forecasting performance, as expressed by V , does not depend on the height of the flood wave.This is considered a reasonable assumption because the warning system performance is based on the exceedence of a flooding threshold only, and not on the prediction of the height of the flood wave.

Case study: White Cart Water
The combination of hydro-economic EAD model with relative economic value is used to estimate flood risk in a small basin in Scotland.The White Cart Water is a river located in the greater Glasgow area and a tributary of the river Clyde.This case study focuses on Overlee gauging station, which is where the White Cart Water enters the city of Glasgow, and the nearby flood warning locations at which flood damage to residential properties has been known to occur.The White Cart Water at Overlee has an upstream area of 106 km 2 , with an average flow in the order of 3.5 m 3 s −1 .The upper parts of the catchment are mainly rural catchment, while the lower catchment is predominantly urban.The White Cart is a very fast responding catchment, with a time of concentration of approximately 3 h.Flooding frequently occurs in the reaches downstream of Overlee, where the river flows through dense residential areas of southern Glasgow.
To mitigate the adverse consequences of flooding, a flood warning scheme is in place.The forecasting and warning system (Cranston et al., 2007;Werner et al., 2009) is operated by the Scottish Environmental Protection Agency (SEPA).It is a statutory requirement to SEPA to issue flood warnings no less than three hours in advance (Werner and Cranston, 2009).The operational forecasting system includes one source of forecast precipitation only (radar now/forecasts) which has a maximum lead time of six hours.While this does not allow the at risk community to take extensive mitigating action, some actions can -and indeed are -taken.Empirical evidence suggests that the initial four hour warning period is associated with the greatest savings (Parker, 1991;Risk Frontiers -NHRC, 2002;Carsell et al., 2004).
Flood risk is estimated for four cases.The two benchmark cases -No Warning and Perfect Forecasts -are investigated first.Subsequently, two imperfect FFWRSs are investigated: one in which deterministic forecasts are used and one in which probabilistic forecasts are used.
Re-forecasting analyses were carried out using an off-line version of an existing forecast production system: FEWS Scotland, which is based on the Delft-FEWS shell (Werner et al., 2004).Deterministic hydrological forecasts for White Cart at Overlee are produced using a sequence of a PDM rainfall runoff model (Moore, 1985), a kinematic wave routing model and an ARMA error correction model (Moore et al., 1990).
Predictive hydrological uncertainty was estimated using Quantile Regression (QR) (Koenker and Bassett Jr, 1978;Koenker and Hallock, 2001;Koenker, 2005;Weerts et al., 2011).QR is a post-processing method that can be used to characterise the relationship between water level forecasts and water level observations in terms of quantiles, or probabilities of (non-) exceedence.The use of a post-processor in near real-time forecasting systems is attractive as the computation time required is limited; in this case, the postprocessor takes less than ten seconds to estimate the predictive distribution.
For the White Cart case study, QR was calibrated using a five year period (1 April 1991 through 31 March 1996), and subsequently validated on a period covering nearly eleven years (1 April 1996 through 20 February 2007).For both calibration and validation periods, records of deterministic water level re-forecasts were constructed using FEWS Scotland.The hydrological model was forced using observed precipitation.While using so-called perfect forcing significantly reduces uncertainty compared to a situation in which precipitation forecasts are used (Werner and Cranston, 2009), this equally affects both probability forecasts and deterministic forecasts.It does therefore not affect the demonstration of the method presented in this paper.
Deterministic water level forecasts from the calibration period were paired with observations and from these two time series, the quantile regression relationship h τ = f (s) was determined for all quantiles τ ∈ (.01,.02, ..., .99).For the validation period, a probabilistic re-forecast was established through application of the quantile regression relationship to In both graphs, the vertical red line indicates the forecast issue time (t 0 ).In this case, it was forecast that there was a 20 % probability of threshold exceedence at t 0 + 6 h .The posterior water level observation (dotted blue line) showed that the threshold was exceeded at this time.
each deterministic forecast to derive water levels corresponding to the 99 quantiles τ ∈ (.01, .02,..., .99).From this discretised predictive probability distribution, the probability of exceedence of the flooding threshold (local datum + 1.5 m) was determined (Fig. 3).This threshold coincides with the water level at which flood damage starts to occur.
3 Case study results

Case 1: no warning
For Overlee gauging station, an 18-year record of 15-min water level observations and a rating curve were available.Observed water levels were rated and from this record the flood duration curve was established.A stage-damage relation was not available and was established.First, the number of properties affected as a function of flood stage at Overlee was estimated.For simplicity it was assumed that inundation depth is linearly correlated with river stage at Overlee, i.e. that an increase in river stage at Overlee leads to a similar increase in river stage at these properties.The damage to individual properties as a function of inundation depth was determined from Penning-Rowsell et al. (2005).Combining the number of properties affected as a result of a level at the Overlee gauging station and the flood damage per individual property yields the flood damage as a function of stage at Overlee.Using the hydro-economic EAD-model, the depthdamage probability distribution was established (black line in Fig 4).From this distribution, the expected annual flood damage can be calculated.In this case, this expected damage (EAD nowarn ) amounts to 394 695 GBP a −1 .

Case 2: perfect forecasts
One of the primary aims of a FFWRS is to reduce flood losses.Flood damage for individual properties can be considered as the sum of damage to building fabric and damage to household inventory.It is assumed that in the White Cart basin, given the relatively short time available for mitigating action, damage to building fabric cannot be avoided and constitutes an unavoidable loss.Carsell et al. (2004) investigated which categories of household items may be saved given a certain length of mitigation time.This information was combined with the stage-damage relationships from Penning-Rowsell et al. (2005), which is conveniently broken down into similar categories.This allows for estimating new stage-damage curves for single residential properties, conditional on the length of mitigation time available.These can be used to determine new stage-damage curves for the White Cart basin, which are subsequently used to plot the probability-damage curves for the Perfect Forecasts case (Fig. 4).Calculating the area below these curves yields the expected annual flood damage, conditional on the presence of a perfect warning system and given a certain mitigation time.These amounts are listed in avoided increase with lead-time as expected, although the relationship is not smooth due to increments in the categories of items being potentially saved at increasing lead-times.
Loss reduction comes at a cost, namely that of flood warning response.In the hydro-economic EAD model, costs can be added to flood damage in the stage -damage relationship (top right quadrant of Fig. 2).This leads to a changed probability -damage relationship, thus yielding new estimates of flood risk which now includes the cost of warning response.Assuming that the response cost may be expressed as a fraction of avoidable losses (Eq.2), resulting flood risk (original flood risk minus loss reduction plus response costs) may be plotted as a function of r (Fig. 5).This shows that for users whose costs are negligible (which means that r ≈ 0), the maximum loss reduction is attained (i.e. that of the Perfect Forecasts case), with an increase of losses avoided as leadtime increases (Eq.2).Flood risk increases with cost-loss ratio; if the cost of warning response approaches the amount of potential loss reduction (r → 1), flood risk approaches original, No Warning levels.For values of r > 1, where the cost of response is larger than the losses avoided, the total flood risk would increase when compared to the case of No Warning.This is not considered here as it would then clearly not be rational to employ a flood warning service.Note that lines for 5-and 6-h lead-times coincide as the potential losses avoided are equal.

Case 3: deterministic forecasts
In reality, the forecasting component of a FFWRS is unlikely to be perfect and predictive uncertainty will result in both missed events and false alarms occurring.For White Cart, the frequency of these was determined using the reforecasting analysis.The available record of precipitation observations (April 1996-January 2007) was used to force the hydrological forecasting model for White Cart.Forecasts were produced four times daily with a maximum forecast horizon of 6 h and paired with their corresponding observations.This information was subsequently used to create contingency tables (one for every lead-time, Table 3).This table shows the number of occurrences of hits h, missed events m, false alarms f and quiets q respectively, adding up to the total number of decisions made N .This is a high number as the re-forecasting analysis covered almost 11 years with a re-forecast being produced four times daily.While this reforecasting frequency seems high, it still causes some sampling issues, as shown by the performance of the 3-h leadtime re-forecasts versus that of the 5-h lead-time re-forecasts: the latter has a better ratio of hits to false alarms than the former.
The information from the contingency tables was used to determine REV as a function of cost-loss ratio and lead-time (Fig. 6).The figure shows that REV for the 1-h forecasts is unaffected by false alarms as none were observed in the re-forecasting period at this short lead-time.This results in the REV being independent of the cost-loss ratio.As there were misses at these short lead-times, the REV is lower than that of the perfect forecast.Longer lead-times all show declining REV with increasing cost-loss ratios.This is due to false alarms which become increasingly expensive with increasing values of r.It can now be seen that the REV for forecasts at 5-and 6-h lead-time no longer coincide -as the uncertainty increases with lead-time, resulting in an increasing number of false alarms and misses.Flood risk in the present case can be calculated by scaling the flood risk estimates from benchmark cases No Warning and Perfect Forecasts with REV, using Eq. ( 10).This gives EAD FFWRS as a function of lead-time and of cost-loss ratio.Flood risks for the Deterministic Forecasting case for all lead-times and all users are shown in Fig. 7.The figure also shows the original flood risk (i.e. from the No Warning case).It can be seen that for users with a cost-loss ratio 0 ≤ r ≤ .8,issuing warnings with 5-h lead-time leads to the lowest flood risk.Users with higher cost-loss ratios benefit most from warnings based on a 1-h lead-time.For these users, false alarms are costly and minimising forecasting uncertainty yields more benefits than a longer mitigation time.
For all lead-times larger than 1 h, the resulting flood risk increases beyond that of the case using No Warning for the higher values of r.This is again attributed to the increasing expense of false warning-response.At 6-h lead-times a much higher residual flood risk is found than at the 5-h lead-times, meaning that considering 6-h lead-time forecasts in making a decision to initiate a warning response is detrimental for values of r 0.75.Clearly this is a result of the lack of additional potential of avoiding losses at this increased lead-time (Table 2), combined with the occurrence of fewer hits and more misses and false alarms.

Decision rules Envelope
Fig. 8. Relative economic value as a function of cost-loss ratio for decisions based on probabilistic forecasts with a 3-h lead-time.The grey lines correspond to the values of V for each of the decision rules P ≥ 0, 0.1, 0.2, ... , 1 (from top to bottom panels).The black line is the envelope of these curves.

Case 4: probabilistic forecasts
For the forecasting sub-system based on probabilistic forecasts, users may choose their own decision rule.This means that they may either raise or lower the probability threshold at which a decision whether or not to initiate a warning response is taken.While probabilistic forecasting and associated decision rules do not affect flood losses that can be avoided at different lead-times, it does affect probabilities of detection and false alarm rates and therefore allows the user to optimise residual flood risk by tuning the costs associated with forecasting uncertainty.
In this case, a hindcast was made using the hydrological model and the QR post-processor.The same hindcasting period and forecasting frequencies as in the Deterministic Forecasts case (Sect.3.3) were used.For every forecast, the probability of exceeding the flooding threshold was determined, and these were paired with the observed threshold exceedences.From these pairs of forecasts and observations, for every decision rule, the number of resulting hits, misses, Original flood risk Decision rules p = {0, .1,.2,..., 1} Envelope Fig. 9. Flood risk as a function of cost-loss ratio for decisions based on probabilistic forecasts with a 3-h lead-time.The grey lines correspond to the values of flood risk for each of the decision rules P ≥ 0, 0.1, 0.2, ... , 1 (from bottom upwards panels).The black line is the envelope of these curves.
false alarms and quiets was determined.Table 4 shows these numbers for forecasts with a 3-h leadtime.For the decision rule "warn if forecasted event probability is equal to or higher than 0 %" -i.e.always issue a warning -the number of hits is equal to the number of observed events (h = o = 15) with the number of false alarms being equal to the number of forecasts made, minus the number of hits (f = N − h = 15 857).At the other extreme, the decision rule "warn if forecasted event probability equals 100 %" results in zero hits, zero false alarms and all events missed.These in turn were used to determine REV as a function of cost-loss ratio and lead-time.Figure 8 shows the REV for forecast with a lead-time of 3 h.Note that the figure shows multiple REV-curves; one for every decision rule, where probability of flooding exceeds 0, .1,.2,..., .9, and 1.The upper enveloping curve is printed in black, showing the optimal decision rule as a function of the cost-loss ratio r.It is assumed that every user will optimises the REV by choosing the decision rule coinciding with their own cost-loss ratio.The procedure to calculate flood risk is identical to that used in the previous case.Figure 9 shows the resulting flood risk.For higher values of r, the increasing cost of response to false alarms reduce the benefit of flood warning, ultimately resulting in a higher residual flood risk than in the No Warning case.The increasing number of false alarms for decision rules with decreasing thresholds compounds this effect.The decision rule with a probability threshold of 1 converges to the same residual risk as for the No Warning case for all r, with the added cost of operating the (useless) FFWRS.

Lead time = 6h
No FFWRS Probabilistic forecasts (envelope) Deterministic forecasts Perfect forecasts Fig. 10.Flood risk as a function of cost-loss ratio, for all lead-times and all cases.

Summary of results
The flood risk estimates for different scenarios are summarised in Fig. 10.The figure contains six plots, one for each lead-time considered.All plots show results from the four cases investigated.The No Warning case results in flood risk values that are independent of either lead-time or forecasting uncertainty and therefore constant for all users.
In case of a perfect FFWRS, increased lead-time results in increased loss mitigation and decreasing flood risk.Maximum loss mitigation, i.e. minimum flood risk, is attained for those users whose actions come at little or no cost (r ≈ 0).For all other users, the costs of mitigating action increases flood risk.If the cost of flood response equals the mitigated losses (r → 1), flood risk is equal to that in the No Warning situation.
Results of the imperfect FFWRS cases show that there is a trade-off between the benefits of loss mitigation and the costs associated with forecasting uncertainty.Both increase with lead-time, while that benefit decreases with increasing costloss ratio.In all cases, the envelope curve of the probabilistic forecasts results in a lower residual risk than for the deterministic forecast, irrespective of the lead-time and cost-loss ratio of the user.As the cost-loss ratio approaches zero, the probabilistic forecast converges to the perfect forecast system.This is in a sense meaningless, as the low cost of response results in probability thresholds being set to zero so that the response decision is positive for every forecast made.This artefact disappears with increasing response costs.For users with high cost-loss ratios, the costs associated with forecasting uncertainty can be so high that the resulting flood risk is higher than it would be if no system were in place.It is interesting to note that the cost-loss ratio at which this occurs www.hydrol-earth-syst-sci.net/15/3751/2011/ Hydrol.Earth Syst.Sci., 15, 3751-3765, 2011 is very similar for both the probabilistic and deterministic forecasting sub-system.

Probabilistic versus deterministic forecasting
The method presented allows for estimating the costs of forecasting uncertainty given different decision rules.Thus, deterministic forecasts and associated decision rules can be compared with probabilistic forecasts and decisions.The analysis shows that when optimising on long-term flood risk, probabilistic forecasts yield higher flood risk reductions than deterministic forecasts.This is due to the fact that a user can choose a probabilistic decision rule that is befitting of the user's cost-loss ratio, thus optimising on expected costs and benefits.In the case of deterministic forecast, this is not possible due to the absence of uncertainty information and therefore a lack of information for risk-based decision-making.
In the application of the method to the White Cart, observed precipitation was used in the forecast re-analysis period.Forecast uncertainty was estimated through Quantile Regression, with the regressions derived based on the deterministic model performance using these data.There is interdependency between the two types of forecasts: the uncertainties in the deterministic forecasts are made explicit in the probabilistic forecast.The main difference, of course, is that these uncertainties remain "hidden" in the case of single value forecasting, thus preventing risk-based decision making.
If the technique for producing probability forecasts would depend on the use of different forcing data than that used for producing deterministic forecasts, this interdependency could well be different.It is not uncommon, for example, for flood forecasting agencies to use a high-resolution deterministic meteorological forecast for a deterministic forecast, and a meteorological ensemble product of lesser resolution for a probability forecast.In that case, the uncertainties could be different and the relative performance of the two cases could be different also.

Limitations and assumptions
The hydro-economic EAD model and the theory of Relative Economic Value are tools that value systems in terms of direct, tangible damage only.Indirect and/or intangible flood damage is not included in the flood risk estimates, nor in the estimates of cost-loss ratios.Notably, there may be a wish to estimate the number of flood casualties and the reduction thereof by flood risk management measures (e.g.Molinari, 2011).Possibly, the model can be adapted to include casualties and other types of flood damage but in the present paper, no attempt to do that has been made as it was deemed to be outside of its scope.
Another limitation to the hydro-economic EAD model is the assumption that direct, tangible flood damage can be estimated as a function of flood depth only.This omits other important determinants such as flow velocity, flood duration and flood water quality.Merz et al. (2010) suggest that flood depth is the most important indicator of flood damage, as is considered here.Penning-Rowsell et al. (cited in Messner et al., 2007) propose a simple method to include additional parameters such as duration of flooding by increasing the damage at a given depth.Other factors can equally be incorporated to create a "compound" depth-damage curve.
In this paper, it was assumed that decisions are based on forecasts only.In reality, forecasters will add an important element to the forecast model output: expert judgement.Very likely, this expert judgement will introduce a probabilistic element to deterministic model outputs.Forecasters will only issue a warning if they think there is a high probability of flooding.In that sense, the deterministic system that is assessed in this paper is a stereotype that may not be easily found in reality.
Flood warning systems introduce costs, including initial costs for designing and implementing a system, recurring costs for operation and maintenance, and variable per-event costs.The approach that is presented in this paper assumes that these costs can all be included in the per-event costs.Alternative attributions of costs may exist though.Possibly, these alternative methods can be included in the method.For example, recurring costs may be included in flood risk estimates by shifting the stage -damage curve to the right.Initial costs can then be included in annually recurring costs.This is in line with best practices on depreciation of assets, where the investment in the warning system is allocated to its expected useful life.
In calculating the benefits of the provision of warning, it would seem that the reduction of losses in this case are modest.These have been derived using only a rough estimate of damage to inundation in the flood warning area downstream of Overlee, and a more complete flood risk assessment would be required to provide more reliable figures.When considering the possible benefits of flood warning, it is important to consider the economy of scale.Operational costs for forecasting are incurred in FEWS Scotland at the national level (Werner et al., 2009), which provides warnings across Scotland.Whilst the costs of modelling increase with every warning scheme considered, it is clear that many costs are shared -thus increasing the relative benefit of flood warning.

Possible implications for policymakers
The present study shows that FFWRS that are based on probabilistic forecasting bring higher benefits than FFWRS that are based on deterministic forecasting.These benefits can only be realised, of course, if forecasting authorities include probability forecasting in their standard operating procedures.In England and Wales, such a move was Hydrol.Earth Syst.Sci., 15,[3751][3752][3753][3754][3755][3756][3757][3758][3759][3760][3761][3762][3763][3764][3765]2011 www.hydrol-earth-syst-sci.net/15/3751/2011/ recently suggested by Pitt (2008).However, the Pitt Review also suggested that "... the Met Office and the Environment Agency should produce an assessment of the options for issuing warnings against a lower threshold of probability".The present study, however, shows that this may not be a good option for all forecast users.Probabilistic forecasting allows for a decision maker to choose a decision rule in terms of the required minimum probability of threshold exceedence.This assumes that the user is capable of optimal decision making in the presence of uncertainty, but also that the cost-loss ratio is known.Especially the latter is not trivial and may be subject to considerable uncertainties.Also, a user's cost loss ratio may change over time and may depend on flood stage and lead-time.
The benefits of FFWRSs depend to a high degree on system efficiency, which consists of a number of factors pertaining to other elements of a FFWRS than its forecasting component.Here, it was assumed that damage mitigation is half of the potential damage mitigation (L a = .5L p ).Note that this affects all "with warning" cases equally.Increasing system efficiency is outside of the scope of the present paper but currently the topic of scientific research (e.g.Parker et al., 2008;Molinari and Handmer, 2011).
The benefits of probabilistic forecasting can only be attained if forecast users apply optimal decision rules, i.e. if they are able to manage predictive hydrological uncertainty.This may pose substantial requirements to decision makers.Possibly, they will have to be trained in decision making.Also, it is likely that a shift to probabilistic forecasting will require forecasting procedures to be adjusted.
The approach that was presented may help a decision maker in prioritising available flood risk management measures.The present paper shows that these may include measures aimed at reducing either the cost of warning response, at increasing the potential loss reduction, or both.For example, increasing the potential loss reduction may be achieved by increasing the efficiency of flood warning (Sect.2.1) through awareness raising or flood response exercises.The flood risk analysis now allows for these non-structural measures to be compared with structural engineering measures.

Open questions and future research
Probabilistic forecasts used in the present study have not been evaluated in terms of reliability or sharpness.Whereas here, the envelope of multiple probabilistic forecasting risk curves was used, it was not checked whether these coincide with optimal decision rules.Should the probability forecasts show poor calibration, this may not be the case.Additionally, while it is known that the value of a FFWRS does not always increase with forecasting accuracy (Murphy and Ehrendorfer, 1987), it is assumed that the value will increase with increasing sharpness.It would be worthwhile to have a clearer idea of what qualities of a forecast need to be improved for maximisation of value.
The benefits of probability forecasting stem from the possibility of tuning a decision rule so that an optimal balance between forecasting lead-time and forecasting uncertainty is attained.This is assumed not be the case in deterministic forecasting as only a single decision rule is deemed possible.Theoretically however, this assumption may be relaxed and warnings may be issued against a single value threshold different from the flood level.This calibration of deterministic warnings may bring identical benefits.
In reality, FFWRS rarely use a single threshold only.Often, a phased warning and response approach is used.These phases may range from an increase in forecasting frequency to evacuation of floodplain residents.In principle, a phased approach will also benefit from a move to probabilistic forecasting.

Conclusions
A method for estimating the benefits of flood forecasting and warning, and comparing this against the benefit of other flood risk reduction measures is presented.The method is based on the established hydro-economic expected annual damage (EAD) model.This model is extended with the concept of Relative Economic Value (REV), which is a metric for verifying probability forecasts in terms of economic benefits relative to scenarios where a forecasting system is either absent, or perfect.This allows the cost of predictive uncertainty in estimating the benefit of an uncertain (or imperfect) flood warning to be considered.The method allows for comparing the benefits of warning systems relying on deterministic, single value forecasts with those using probability forecasts.In addition, the method may be used to estimate flood risk reduction through improving flood forecasts, e.g. by using more reliable forcings, better models, improved model parametrisation and/or data assimilation.
In the probabilistic case, the probability threshold at which a response is initiated can additionally be optimally chosen as a function of the cost-loss ratio of the forecast user.As uncertainty can be expected to increase with lead time, the method allows an optimal forecast lead-time to be determined, based on the minimisation of long-term flood risk The method is applied in a case study to the White Cart Water, a small catchment on the outskirts of Glasgow, Scotland.In this case study it is shown that: -Using probability forecasts (in combination with the optimal warning rule) results in lower values of residual flood risk when compared to using deterministic, single value forecasts.This is noted throughout different lead times and cost-loss ratios of the user of the forecast.
-The optimal lead-time for warning is not necessarily equal to the longest lead-time that can be provided by the forecasting system, but that it is a function of the cost-loss ratio of the user of the forecast, as well as the uncertainty of the forecast.

Fig. 2 .
Fig. 2. Schematic representation of the hydro-economic EAD-model.The bottom left panel shows the probability distribution of flow rates.The stage-discharge relationship is shown in the top left panel, and flood damage curve in the top-right quadrant.These three relationships yield the damage-probability curve (bottom right panel).The figure shows how a flood risk management measure affects flood risk, with the ex-ante situation as a solid, and the ex-post as a dotted line.

Fig. 3 .
Fig. 3. Sample probability forecast as produced by the research version of the forecast production system FEWS Scotland.Two graphs are shown: a discretised predictive probability distribution of water levels at quantiles τ ∈ (.01,.05, .10,.25,.50,.75,.90,.95, .99)(top panel) and the probability of exceedence of the flooding threshold (bottom panel).In both graphs, the vertical red line indicates the forecast issue time (t 0 ).In this case, it was forecast that there was a 20 % probability of threshold exceedence at t 0 + 6 h .The posterior water level observation (dotted blue line) showed that the threshold was exceeded at this time.

Fig. 5 .
Fig. 5. Flood risk in the Perfect Forecasts case, as a function of cost-loss ratio and lead-times.This flood risk includes unavoidable flood damage and the cost of flood warning response.

Fig. 6 .
Fig. 6.Relative economic value as a function of cost-loss ratio and lead-time in the Deterministic Forecasts case.

Fig. 7 .
Fig. 7. Flood risk as a function of cost-loss ratio and lead-time in the Deterministic Forecasts case.

Table 1 .
Contingency table.The consequences of the items listed are in brackets.

Table 2
. The EAD column shows the flood risk reduction (losses avoided) achieved by the (perfect) warning system.The table shows that losses Fig. 4. Hydro-economic EAD model for the No Warning (black) and Perfect Forecasts (grey) cases.Reduced damage as a result of flood warning response results in reduced expected damage; from right to left panels, grey lines show damage curves as a function of increasing mitigation times from 1 to 6 h.Resulting flood risk is listed in Table2.

Table 2 .
Loss reduction in terms of expected annual damage (response costs not included).

Table 3 .
Performance of the FFWRS based on deterministic forecasts, expressed in the elements of a contingency table.

Table 4 .
Performance of a warning system based on probabilistic forecasts, expressed in the elements of a contingency table.This table pertains to decisions based on forecasts with a 3-h lead-time.