Interactive comment on “ Hydrological drought forecasting and skill assessment for the Limpopo river basin , Southern Africa ” by P . Trambauer

The paper presents hydrological seasonal prediction experiment using three different driving meteorological inputs (dynamic ECMWF seasonal forecast, ESP and ENSO conditioned ESP) for Limpopo River basin. The study in general follows correctly a common methodology applied for such studies, however does not provide detail information about some important steps of the whole process (see. bellow). The text should be more inclusive to provide reader with all important information on methodology without simply referring to other existing studies; e.g. the results of NS criterion of hydrological model calibration should be stated (authors only refer to another study of the same team P9978 "In these stations the performance of the hydrological model is found to be satisfactory based on evaluation measures and ranges proposed by Moriasi et al. (2007). These results are presented by Traumbauer et al. (2014)."). In


Introduction
Climate change studies show evidence of an intensification of the global water cycle (Huntington, 2006;IPCC, 2007;Hansen et al., 2012;Trenberth, 2012;Coumou and Rahmstorf, 2012), with extreme events including floods and droughts expected to become more frequent.The UNISDR (United Nations Office for Disaster Risk Reduction) Hyogo Framework of Action 2005-2015(UNISDR, 2005) describes early warning systems and action plans triggered on the issuing of a warning as one of the most effective strategies to mitigate the impacts of natural hazards.Operational forecasting of streamflow to inform early warning is already commonplace in several parts of the world, but the main focus is often on flood prediction.Operational forecasting of streamflow for drought prediction has to date not been applied as widely, despite the widespread recognition of the relevance and importance of drought forecasting in the research community.
There are several Drought Early Warning Systems (DEWS) currently in existence in the world, though due to the complexity of drought these are arguably less developed P. Trambauer et al.: Hydrological drought forecasting and skill assessment than many flood early warning systems.Grasso (2009) reports that only three institutions provide information on the occurrence of major droughts at the global scale: FAO's Global Information and Early Warning System on Food and Agriculture (GIEWS), the Humanitarian Early Warning Service (HEWS) operated by the World Food Programme (WFP), and the Benfield Hazard Research Centre at University College London.
In the United States the US Drought Monitor (http:// droughtmonitor.unl.edu/) was set up in collaboration between the US Department of Agriculture (USDA), NOAA, the Climate Prediction Centre, and the University of Nebraska.It provides insight to current drought conditions and impacts at the national and state level through an interactive map, presenting multiple drought indicators combined with field information and expert input.It also includes 6-10day outlooks and monthly and seasonal forecasts of precipitation, temperature, soil moisture and streamflow.The National Weather Service's National Center for Environmental Prediction's (NCEP) also has a (multi-model) drought monitoring system, as well as a seasonal hydrological forecasting system running at the Environmental Modeling Center (Ek et al., 2010).Additionally, the North American Multi-Model Ensemble (NMME), which became an experimental real-time system in August 2011, is mainly focused on seasonal prediction of meteorological drought (Kirtman et al., 2013).
In Europe the European Commission Joint Research Centre (JRC) has established the European Drought Observatory (EDO, http://edo.jrc.ec.europa.eu/),which includes an interactive map viewer with drought-relevant information.It includes real-time maps of different drought indicators, including the standardised precipitation index (SPI), snow and soil moisture anomaly, and vegetation productivity anomaly.These indicators are combined in an overall indicator that is used to provide warnings and alerts.A 1-week forecast of the expected soil moisture anomaly is also provided.The Beijing Climate Center (BCC) of the China Meteorological Administration (CMA) similarly monitors the development of drought across China, with maps on current drought conditions being updated daily on their website.
The FEWS Net (Famine Early Warning Systems Network) for eastern Africa, Afghanistan, and Central America reports on current famine conditions, including droughts, by providing monthly bulletins that are accessible on the FEWS Net web page.However, a drought forecast is not provided.Other drought warning systems over Africa include the Botswana national early warning system (EWS) for drought (Morgan, 1985) and the Regional Integrated Multi-Hazard Early Warning System for Africa and Asia (RIMES).In the latter a drought early warning system is being adapted to identify climate and water supply trends in order to detect the probability and potential severity of drought (RIMES, 2014).
Advances regarding drought early warning systems in Africa in the last few years are remarkable.There is an in-creasing availability of drought monitoring and forecasting tools for decision making that can provide real-time monitoring and forecasting of drought across the continent.The Land Surface Hydrology Group at Princeton University, USA, has recently established an African Flood and Drought Monitor (http://stream.princeton.edu/) with support from the International Hydrology Program of UNESCO.The system provides near-real-time monitoring of land surface hydrological conditions based on the Variable Infiltration Capacity (VIC) model.The monitor is updated every day at 2 days behind real time.The database provides the daily conditions of precipitation, temperature, wind speed, soil moisture, evaporation, radiation, and different components of runoff in the continent, as well as historic hydrological records in eastern, southern and western sub-regions for up to 10 antecedent years, and derived products such as current drought conditions.They also provide precipitation, temperature and SPI forecasts (Sheffield et al., 2014).Recently, Barbosa et al. (2013) developed a pan-African map viewer for drought within the framework of the DEWFORA project, following the main features of the earlier developed EDO.The African Drought Observatory (ADO) is a web application hosted by JRC (http://edo.jrc.ec.europa.eu/ado/ado.html)that provides historical and near-real-time monitoring information, as well as seasonal forecasts describing meteorological, agricultural and hydrological droughts (Barbosa et al., 2013).Yuan et al. (2013) applied the NCEP's Climate Forecast System version 2 (CFSv2) combined with the VIC land surface model for seasonal drought prediction over Africa.They used both the SPI and soil moisture as indices and the Brier skill score (BSS) to assess the probabilistic drought hindcast for 1982-2002.Their results show relatively good skill in the dry season but only limited skill in the rainy season.They indicate that CFSv2 precipitation is correlated with the observed precipitation over southern Africa, but only accounts for 44-45 % of the variance of observations.They point out that for two extreme droughts CFSv2 predicted neutral conditions or only a weak anomaly.Our study focuses on the Limpopo River basin and follows a similar type of analysis, although it does so at a higher resolution and in a more detailed manner.Additionally, we present different skill scores for different hydrological drought indicators during the rainy season, and compare different forecasting systems in the basin.We focus on assessing the skill of the forecast in predicting indicators that are meaningful to the local end users in the basin.
The semi-arid Limpopo River basin, located in southern Africa, has experienced severe droughts in the past, which have led to crop failures, high economic losses and the need for humanitarian aid.An effective drought early warning system for this basin is of prime importance.Current practices for drought forecasting in the Limpopo River basin involve three forms of seasonal climate forecasts ranging from regional to local scales: the Southern Africa Regional Climate Outlook Forum (SARCOF) climate outlooks, seasonal cli-mate outlooks prepared by meteorological departments, and forecasts based on local knowledge applied in rural communities.Despite these seasonal forecasts being available in the basin, farmers seem to prefer to rely on drought forecasting systems based on indigenous and traditional knowledge.Such forecasts include signs in (i) the sun, moon and wind; (ii) trees and plants; and (iii) insects, birds and animals (DEWFORA, 2013).For seasonal forecasts to be accepted by the local community there are several challenges that need to be addressed.End users should receive the information in a suitably understandable format at the time they need it for the forecast to be useful.The highly technical information that is typically contained in the forecasts should then be translated to a comprehensible form before being disseminated and delivered to decision makers and farmers.Moreover, end users should be involved in the product verification by providing feedback to the forecasters (DEWFORA, 2012).
Seasonal hydrological drought forecasts aim for high hydrological predictability at a seasonal timescale.Shukla et al. (2013) quantified the contribution of a good representation of initial hydrologic conditions (IHCs) and seasonal meteorological forecast (MF) to seasonal hydrological predictability at different forecast dates and lead times (1, 3, and 6 months) globally.They quantified the contributions of two components of the IHCs (soil moisture and snow water content) through ensemble streamflow prediction (ESP) and reverse-ESP.Their results show that for the region of the Limpopo River basin the MF dominates the hydrological predictability during the wet season (forecasts starting in October and January) for almost every lead time considered.Only for the 1-month lead time forecasts issued in October did the IHCs appear to some extent to have a higher influence.For the dry season the IHCs dominate the hydrological forecast at all lead times.These results suggest that to improve the seasonal hydrologic forecast skill in the Limpopo River basin, efforts should focus on improving the MF.However, the contribution of other IHCs (surface water and groundwater level) to hydrological predictability should also be assessed.Yossef et al. (2013) also investigated the relative contribution of initial conditions and meteorological forcing to the skill of the global seasonal streamflow forecasting system FEWS-World, using the global hydrological model PCR-GLOBWB (PCRaster Global Water Balance).They use ESP and reverse-ESP to determine the critical lead time for different locations at which the importance of the initial conditions is surpassed by that of the meteorological forcing.They indicate that for semi-arid regions such as the Limpopo Basin the initial conditions do not contribute much to the skill given the high sensitivity of the runoff coefficient to rainfall variability.This would suggest that the predictability in semi-arid basins such as the Limpopo using ESP is limited, with seasonal meteorological forecasts potentially offering better skill.
In this study we introduce three dynamic forecasting systems based on a distributed hydrological model for the seasonal prediction of hydrological droughts for the semi-arid

Methods and data
The approach followed in this study is summarised in Fig. 1.It starts with obtaining the meteorological seasonal forecast and preprocessing the data.This is then used to force the hydrological model (embedded in the Delft-FEWS forecasting shell; Werner et al., 2013), thus obtaining seasonal forecasts of streamflow and other hydrological variables.considered to be at serious risk of starvation (DEWFORA, 2011).The annual rainfall in the basin averages some 530 mm yr −1 , though the spatial variation is significant, ranging from 200 to 1200 mm yr −1 .Rainfall occurs mainly in the austral summer months (October-April; LBPTC, 2010).As is common with semi-arid and arid basins, runoff coefficients in the Limpopo are very low, being only 4.3 % for the naturalised discharge and a mere 1.7 % for the observed discharge at Chókwe station in Mozambique, which is the most downstream station considered in this study.If abstractions are included then only 23 of the 539 mm yr −1 precipitation average for the basin upstream of Chókwe runs off.Consequently, hydrological modelling in the Limpopo Basin is extremely challenging.Even a small error in precipitation or evaporation estimates could result in quite a large error in runoff estimation.Moreover, the uncertainty in the rainfall input could easily be larger than the runoff coefficient (4.3 %) of the basin.Fig. 3 shows the location of selected runoff stations and reservoirs in the Limpopo Basin.

Regional hydrological model
A finer-resolution version (0.05 × 0.05 • ) of the 0.5 × 0.5 • resolution global PCR-GLOBWB hydrological model is used.This is a continuous-time simulation, process-based distributed model applied on a raster basis.PCR-GLOBWB is in many ways similar to other global hydrological models but it has several improved features, such as improved schemes for sub-grid parameterization of surface runoff, interflow and baseflow, a kinematic wave-based routing for the surface water flow, dynamic inundation of floodplains and a reservoir scheme (van Beek and Bierkens, 2009;van Beek, 2008).The model is set up for the Limpopo Basin with a spatial resolution of 0.05 × 0.05 • and the simulation is carried out with a daily time step.As the scope of this study is on the skill of the hydrological forecast, reservoirs are considered in a simple way.Cells with reservoirs in the model are considered as having a maximum storage volume.Releases to irrigation are taken into consideration as a fixed monthly value and subject to availability, and the reservoir will spill when full.The reservoirs in the basin are mainly used for irrigation.For a more detailed description of the model setup for the Limpopo River basin the reader is referred to Trambauer et al. (2014b).

Delft-FEWS shell
The hydrological model is embedded in the Delft-FEWS (Flood Early Warning System) open shell for forecasting purposes.The shell provides a sophisticated collection of modules designed for building a hydrological forecasting system customised to the specific requirements of an individual organisation.The philosophy is to provide an open shell for managing the data handling and forecasting process.This shell incorporates a comprehensive library of general data handling utilities, allowing a wide range of external models to be integrated in the system through a published open interface.This allows existing simulation models and data streams to be incorporated into a comprehensive and reliable forecasting system (Werner et al., 2013).

Reference run for the period 1979-2010
The hydrological model is run in simulation mode for a 32-year period    Dee et al. (2011).The ERA-Interim precipitation data used in this study was corrected using the GPCP (Global Precipitation Climatology Project) v2.1 product to reduce the bias when compared to measured products (Balsamo et al., 2010).The GPCP v2.1 data is a monthly climatology provided globally at 2.5 × 2.5 • resolution, covering the period from 1979 through to September 2009.It combines the precipitation data available from several sources (satellite data, rain gauge data, etc.) into a merged product (Huffman et al., 2009;Szczypta et al., 2011).
From September 2009 to December 2010, the mean monthly ERAI precipitation was corrected using a mean bias coefficient based on the climatology of the bias correction coefficients that were established for the period 1979-2009.While this only corrects for systematic biases, this was the only option available at the time, as a new version of GPCP (version 2.2) was not available.This corrected version of precipitation was also used in the production of the ERA-Interim/Land data set (Balsamo et al., 2015).
In addition to the precipitation, other meteorological parameters from the ERA-Interim reanalysis data that are used to force the model include the 2 m daily temperature (minimum, maximum and average).Temperature data is used for the computation of the reference potential evaporation that is required to force the hydrological model.In this study the Hargreaves formula is used.This method requires less parameterization than the Penman-Monteith formula, though it has the disadvantage that it is less sensitive to (uncertain) climatic input data, with a possible reduction of the dynamics and accuracy of the potential evaporation as a consequence.However, this also means that it is less sensitive to errors in climatic inputs (Hargreaves and Allen, 2003) that are inherent to any meteorological forecast.Moreover, the choice of the method used for the computation of the reference potential evaporation was shown to have minor effects on the results of the actual evaporation for southern Africa, where actual evaporation is dominated by soil moisture availability (Trambauer et al., 2014a).The ERA-Interim data for the 32-year period from 1979 to 2010, corrected using the GPCP v2.1 data set, are converted to the same spatial resolution as the continental-scale version of the PCR-GLOBWB model.ERAI is archived on an irregular grid (reduced Gaussian) with an approximate resolution of 0.7 • over the domain.The data is downscaled from the ERAI grid to the original 0.5 • model grid using bilinear interpolation and assumed to be constant over the 0.5 • grid cell.No further downscaling of the meteorological forcings is carried out.

Initial conditions
The reference run provides the initial conditions for all forecasts.Initial conditions at the beginning of each month are saved in the Delft-FEWS database, and subsequently used as "warm states" to start the forecasts when doing the retroactive forecast (also referred to as hindcasts).

Time period of the simulations
An ensemble of meteorological hindcasts is first tested for the summer rainfall season over southern Africa for the period 1981-2010.Seasonal forecasts in this study are issued for only 7 months of the year so as to capture the rainy season and main runoff season (meaning the there are 5 months where we do not issue a forecast).The predictive skill for drought is expected to be higher during the dry season and lower during the wet season given that the hydro-climate has a longer persistence during the dry season (Yuan et al., 2013).Yuan et al. (2013) show the high contrast in skill between the dry and wet seasons in southern Africa.
In the hindcast, the first forecast of each season is issued in August and includes the seasonal (6 months) forecast from August to January.The forecast is updated at the beginning of each month from September to February.The last forecast of the season is issued in February, covering the period from February to July (see Fig. 4).All simulations are done at a daily time step.

Seasonal forecasting systems
All three forecasting systems considered use the same hydrological model of the basin, but are forced with different meteorological forecasts.In the first system (FS_S4) the PCR-GLOBWB hydrological model is forced with the output of a global atmospheric model, the ECMWF seasonal forecast system S4 (atmosphere-ocean coupled).The second forecasting system (FS_ESP) is based on the ESP (Day, 1985) procedure.In the ESP procedure the ensemble meteorologi-P.Trambauer et al.: Hydrological drought forecasting and skill assessment cal forecast is generated with re-sampled historical meteorological data.The hydrological model is then forced with this re-sampled data.A third system (FS_ESPcond) is proposed given that the ENSO has a clear influence on the interannual climate variability over the Limpopo River basin (Landman and Mason, 1999).This is equivalent to the second system but the weights of the ESP ensemble members are conditioned on the ENSO signal (Oceanic Niño Index, ONI).This is explained in full in Sect.2.2.3.

Meteorological ensemble forecasts
Seasonal meteorological forecasts from the most recent seasonal forecasting system at ECMWF (S4) are used to force the hydrological model.The S4 ensemble seasonal forecasts are initialised on the 1st of each month and the ensemble is generated by perturbations in the initial conditions and by the use of stochastic physics in the atmosphere during the model integration (out to 6 months lead time; Molteni et al., 2011).The atmospheric resolution is about 79 km with 91 vertical levels, and is fully coupled with an ocean model with a horizontal resolution of 1 • .S4 has been in operational use since November 2011, issuing 51 ensemble members with 6-month lead time.A hindcast set is provided for calibration and verification purposes, covering a period of 30 years  with the same configuration as the operational forecasts but with only 15 ensemble members.Molteni et al. (2011) presents an overview of S4 model biases and forecasts performance, and Dutra et al. (2013Dutra et al. ( , 2014) ) present an evaluation of S4 in seasonal forecasts of meteorological droughts.They found that S4-derived meteorological drought forecasts over southern Africa were skilful with up to 4-months lead time for SPI-6 in April.In the setup of FS_S4, the hydrological model is forced with the re-forecasts of the ECMWF seasonal system S4, with 15 ensemble members.A (hydrological) re-forecast is made to coincide with the 1st of each month in the 30-year hindcast set.Precipitation inputs to the hydrological model are accumulated from the 6-hourly S4 model values, while evaporation was calculated using the daily maximum and minimum temperatures directly archived by the meteorological model.

Climatological bias correction of seasonal forecasts of precipitation
Mean biases and drifts in the seasonal forecasts of precipitation can have a detrimental influence on the hydrological forecasts.Therefore, a simple climatological bias correction, based on monthly means, is applied to the seasonal forecasts in the form where P and P are the original and corrected seasonal forecasts of precipitation, respectively, α is a multiplicative cor-rection factor and the subscripts "m" and "l" are the calendar month (1-12, of the initial forecast date) and lead time (0-5 months), respectively.The correction factor is given by the ratio where P base m * is the climatological long-term mean of precipitation of the base data set for a particular calendar month m * (m * = m+l), and Pm,l is the long term ensemble mean of the forecasts for a particular month "m" and lead time "l".The base data set used was ERA-Interim corrected with GPCP to be consistent with the baseline simulation period.The correction factor α is limited to a reasonable range (0.1-10), and is linearly interpolated from monthly values to daily values by assuming that it corresponds to day 15 of the particular month.Equation ( 1) is applied to the daily precipitation values.This is a simple bias correction that only guarantees that the mean forecast climate is similar to the climate of the base data set.It does not address other problems of the forecasts, common to all coupled atmosphere-ocean models, such as interannual variability, ensemble spread or daily variability.

ESP meteorological forecasts (FS_ESP)
A widely used approach to seasonal forecasting is the ESP procedure.ESP predicts future streamflow from the current initial conditions (warm state) in the hydrological model with re-sampled historical meteorological data (ERA-Interimcorrected with GPCP-observed meteorology from the last 31 years in this study).The procedure assumes that meteorological events that occurred in the past are representative of events that may occur in the future (Day, 1985).Although ESP is normally used in the absence of a seasonal forecast, in this study we use it to compare the skill of the FS_ESP with that of the FS_S4.Moreover, a comparison of these two forecasts may give an indication of what influences the predictability.ESP represents forecast uncertainty due to boundary forcing uncertainties only (Wood and Lettenmaier, 2008) and thus allows measuring the skill that can be expected only from initial states.In the FS_ESP hindcast, the sample of the year in which the forecast starts is excluded from the ensemble to allow for a fair estimate of the forecast uncertainty.The FS_ESP therefore includes 30 (31 minus 1) years in the ensemble.

Conditional ESP meteorological forecasts (FS_ESPcond)
ENSO is clearly related to interannual climate variability over the Limpopo River basin.In southern Africa meteorological droughts tend to happen in the December-March rainy season after onset of an El Niño event (Thomson et al., 2003).However, it is not always the case that this happens.Thomson et al. (2003) recorded a 120 % increase in probability of drought disaster in the year after an El Niño onset.
To account for the relationship between ENSO and the occurrence of drought, this system is similar to FS_ESP but the weights of ensemble members sampled through the ESP procedure are conditioned on the ENSO signal.
We use the post-ESP weighting technique described in Werner et al. (2004).This approach uses the El Niño-3.4 index averaged over the 3-month-period immediately prior to the issue date of the forecast to weight ensemble members from ESP.The technique is summarised here for the forecast of the 6-month standardised runoff index (SRI-6).
1. Compute a vector (X) of absolute differences (x i ) between the value of the Niño-3.4index (ONI) in the forecast year and those of all the other years and sort the vector (X) from lowest to highest.
The sorted vector (ℵ) is 2. Compute a vector of weights (W ) for each member of the ESP ensemble by defining two parameters: a distance-sensitive weighting parameter (λ) and a parameter (α) that defines the k nearest neighbours used to calculate the weight of each member.Higher λ gives more weight to ensemble members with values of ONI closer to that of the forecast year.Higher α restricts attention to the n/α elements in the sorted vector.The ensemble member with the same year as the forecast year is assigned a weight of zero.
The parameters λ and α can be optimised for each case study or subbasin.The case with λ = α = 1 is the traditional equal weighting scheme applied to ESP forecasts, with all ensemble members considered to have equal weight.If α = 1 and λ varies, all ensemble members are considered, but these have non-zero weights that depend on the absolute distance between the ONI of the forecast year and the ONI of the year of the ensemble member.If λ = 1 and α varies only the nearest k ensemble members to the forecast year are considered in the ensemble, but they are all weighted equally.This case is similar to the approach applied by Hamlet and Lettenmaier (1999) for the Columbia River, where they restricted the ensemble members to those years that were similar in terms of the ENSO phase and the Pacific decadal oscillation phase.However, this restriction may result in ensembles with only few members, resulting in forecasts that are very sensitive to sampling errors (Brown et al., 2010).In the last case, where both α and λ vary, weights are assigned only to the k nearest ensemble members based on the distance of the index to the index of the forecast year (Werner et al., 2004).Werner et al. (2004) found this last case where both α and λ vary to show the best improvements for forecast skills.
For the FS_ESPcond we chose to keep the parameters constant (λ = 2 and α =1 ) given that the optimal selection of parameters would vary for each subbasin.Performing an indepth selection of parameters for each subbasin is out of the scope of this study.Here we use λ = 2 and α = 1, meaning that all ensemble members have a non-zero probability of being included in the ensemble, with that probability based on the distance between the ENSO indexes and the distance sensitive weighting parameter (linear for λ = 2).For each forecast start date, we construct an ensemble meteorological forecast of 30 members to be consistent with FS_ESP.The selection of the members is based on a resampling with replacement procedure given the probability assigned to each member.From the 30 possible ensemble members to be included, those with an ONI index closer to that of the forecast year have a higher probability of being included in the ensemble.This means that some ensemble members are included more than once, and some are not included at all.The ONI indexes for the period 1979-2010 were retrieved from NOAA (2014).
We also use this procedure for the forecast of SRI-4 (JFMA SRI).FS_ESPcond always uses the latest ONI index available prior to the start date of the forecast.This means that for the forecast issued in January, which corresponds to a 3-month lead time, FS_ESPcond uses the ONI values for October, November and December (OND).Similarly, for the forecast issued in December, which corresponds to a 4-month lead time, FS_ESPcond uses the SON (September, October, November) ONI, and the forecast issued in November (5month lead time) makes use of the ASO (August, September, October) ONI.

Skill scores
Standard verification skill scores are selected to measure the skill of the forecast ensembles in predicting drought indicators.In this study we use the SRI for the characterisation of hydrological droughts.This indicator is explained in the following section.Forecasts are verified against the reference run and the resulting skills are established relative to sample P. Trambauer et al.: Hydrological drought forecasting and skill assessment climatology.Cloke and Pappenberger (2008) recommend the use of several verification measures in the same analysis so that the quality of the forecast can be assessed rigorously.We selected three verification scores that measure slightly different properties of the forecast skill.The ROC curve measures discrimination but not bias, the rank histogram measures reliability or bias, and the Brier score (BS) accounts both for reliability and sharpness (Renner et al., 2009).
The ROC (relative operating characteristic, or receiver operating characteristic) diagram measures the ability of the forecast to discriminate between two alternative outcomes.It plots the hit rate or probability of detection (POD) versus the false alarm rate or probability of false detection (POFD).It is not sensitive to bias in the forecast, so it says nothing about the reliability.It is conditioned to the observations.In summary, it indicates the ability of the forecast to discriminate between events and non-events given a certain event threshold (WWRP/WGNE, 2013).The area under a ROC curve (ROCS) is used as a score.ROCS can take values from 0 to 1, with a value of 0.5 indicating no skill and a value of 1 representing a perfect score.Values lower than 0.5 indicate negative skill.ROC curves measure how good forecasts are in the context of a very simple decision-making model, and are thus better suited to measure how good forecasts are from the perspective of the user than many other commonly used measures (Tveito et al., 2008).
The BS ([0-1]) measures the mean squared probability error and represents the magnitude of the probability forecast errors, with a perfect score of zero.The BSS ([−∞ to 1]) measures the improvement of the probabilistic forecast relative to sample climatology and indicates what the relative skill of the probabilistic forecast is over that of the climatology, in terms of predicting whether or not an event occurred (WWRP/WGNE, 2013).
The rank histogram is used to evaluate whether the forecast ensembles are from the same underlying population as the observations, which implies that the observed would have the same probability of occurrence as any of the ensemble members.This would result in a uniform distribution in the histogram that plots the frequency of the rank of the observation in the ensemble, while deviations from the uniform distribution reveal deficiencies in ensemble calibration or reliability (Wilks, 2011).

Standardised runoff index
The hydrological drought indicator SRI follows the same concept as the SPI and is defined as a "unit standard normal deviate associated with the percentile of hydrologic runoff accumulated over a specific duration" (Shukla and Wood, 2008).To compute SRI the runoff time series is fitted to a probability density function (a gamma distribution) and the function is used to estimate the cumulative probability of the runoff of interest for a specific month and temporal scale.The cumulative probability is then transformed to the stan-dardised normal distribution with mean of 0 and variance of 1 (Shukla and Wood, 2008).

Skill assessment
Forecasted streamflow is transformed to the hydrological drought indicator SRI and forecasts of drought are analysed by considering drought conditions to occur for SRI ≤ −0.5 (mild to moderate drought).The value of −0.5 was chosen as it corresponds to the 30th percentile in runoff and it is therefore a good compromise between not capturing all negative anomalies and having a sufficient amount of samples for the analysis.The forecasting system is thus evaluated on the skill of predicting SRI falling below the −0.5 threshold.
However, as we also want to analyse the ability of the system to forecast distributed variables (for agricultural droughts) and water levels in the reservoirs (for irrigation curtailments), we also evaluated the skill of the forecast system in predicting these variables.

Estimating uncertainty in the skill scores
Given the small sample size resulting from applying the verification over the 30-year hindcast period, a bootstrap approach is used to estimate the confidence intervals around the ROCS.The idea behind the bootstrap is to treat a finite sample at hand as similarly as possible to the unknown distribution from which it was drawn, which in practice leads to resampling with replacement (Wilks, 2011).The uncertainty of the ROCS is estimated by applying a bootstrap resampling with replacement procedure.
For the FS_S4 and FS_ESP forecasts, we randomly replace (allowing repetition) the original forecast and verification pair to produce a new sample of the same size as our original sample.We then calculate the ROCS from the new sample.We repeat this procedure to create 1000 new samples from which we generate an empirical distribution of the ROCS.The 90 % confidence interval is estimated from the 5th and 95th percentiles of this empirical distribution.
For the FS_ESPcond the bootstrap procedure follows the same theory but is computed slightly differently.In this case the bootstrap is achieved by recreating the ensemble forecasts for the hindcast period 1000 times based on the computed probability vector and computing the skill score from each created ensemble.
A limitation of this bootstrap procedure is that statistics computed from discrete bootstrap samples may differ from the ones based on continuous data, and this might lead to overestimation of the confidence.However, this method is widely used in the literature (Dutra et al., 2014;Friederichs and Thorarinsdottir, 2012;Wilks, 2011) to estimate confidence intervals as it does not require assumptions on the distribution.

Assessing spatial hydrological indicators
ROCS and BSS are computed for the spatially distributed indicator root stress (RS) to assess the skill of the forecast in predicting agricultural drought indicators.The RS is an indicator of the available (or the lack of) soil moisture in the root zone, which can be calculated for each grid cell.The RS varies from 0 to 1, where 0 indicates that the soil water availability in the root zone is at field capacity and 1 indicates that the soil water availability in the root zone is at wilting point and the plant is under maximum water stress.For each grid cell, a drought is defined to occur when the root stress is higher than the 70th percentile of the observed values for that month.An advantage of defining the threshold as a percentile of the observed sample as proposed by Roulin (2007) is that it assures a sufficiently large enough number of events to verify and also allows for comparison of verification statistics at different locations (Renner et al., 2009).
In addition to indicators such as RS, it is interesting to evaluate the skill of the model in predicting indicators that are meaningful to the end users in the basin.Irrigation is the major water use in the Limpopo Basin.The amount of water made available to the irrigation sector may, however, be restricted depending on the water level in the reservoirs in the basin as a percentage of their full capacity (DWA, 2013).The forecasted anomaly of the water level in the reservoir is a decision variable that can give an indication to the water managers of the percentage of irrigation demand that can be covered during the season.
An analysis of the historical time series of water level for the Tzaneen reservoir together with the curtailment rules of the reservoir (DWA, 2013) indicate that a 20 % curtailment to the irrigation sector is applied when water levels in the reservoir fall below the 50th percentile in the water levels (in percentage of the capacity of the reservoir).Similarly, a 65 % curtailment to the irrigation sector is applied when water levels in the reservoir fall below the 37.5th percentile and a 90 % curtailment in the irrigation sector along with a 30 % curtailment in the urban sector when the water levels are below the 12th percentile.ROCS and BSS are then computed to assess the skill of the forecast in predicting the water levels in the reservoirs to be lower than these threshold percentages of the full capacity.Although the actual operation of the reservoirs is quite a bit more complex, this can be interpreted as an assessment of the skill of the forecast in predicting curtailments to the irrigation sector.

Results
The following section outlines the results when applying the different types of forcing to the hydrological model over the 30-year hindcast period from 1981 to 2010.The analysis is carried out for different verification periods and lead times as the forecast quality may vary significantly with temporal scales and lead times.While the rainy season in the Limpopo River basins spans from October to March, the main rains typically take place from November to February.The main runoff season and the high runoff season, however, lag behind the rainy season by 1 or 2 months, occurring in general from December to May and from January to April respectively (see Fig. 4).

Skill of seasonal streamflow prediction
This section presents the skill expressed in the selected skill scores of the seasonal streamflow prediction for the three forecast systems described (FS_S4, FS_ESP, and FS_ESPcond) for Station 24 (Chókwe), Station 1, Station 18 and Station 20 in the Limpopo River basin (see Fig. 3 for the station locations).Station 24 is the one with the largest drainage area in the basin with available discharge data.Four stations (highlighted in Fig. 3) with diverse drainage areas were selected to assess the influence of the spatial scale and forecast location on the quality of the forecasts.Table 1 presents the main characteristics of these stations, such as drainage area, mean annual runoff and observed runoff coefficient (RC = runoff/precipitation).In these stations the performance of the hydrological model is found to be satisfactory based on the evaluation measures and ranges proposed by Moriasi et al. (2007), which comprise the Nash-Sutcliffe efficiency (NSE), and the ratio of the root mean square error to the standard deviation of the measured data (RSR).The coefficient of determination (R 2 ) is also included.These results are presented by Trambauer et al. (2014b) and are summarised in Table 1.
Figure 5 (upper plots) presents the ROC diagram for the 6-month SRI-6 ≤ −0.5.For calculating the SRI-6 the verification period is from December to May and the SRI-6 value is recorded at the end of the period in May.The figure shows three of the four stations considered, for a lead time of 5 months (the forecast is issued in December).December is the only start time of the forecast that captures the whole 6-month main runoff season (from December to May) in the seasonal forecast.The ROC diagram for Station 18 is not presented given that it has a similar behaviour to Station 1.The ROC curves are presented for each forecasting system, and the ROC of FS_ESPcond is represented by the ensemble that results in the median ROCS.Results from the FS_ESPcond show for all stations a narrower 90 % confidence interval when compared with the other two forecasting systems considered (see middle and lower plots in Fig. 5), thus suggesting that FS_ESPcond is more robust.Histograms of ROCS for FS_ESP are not shown as these are similar to those of FS_S4.
The ROCS of the FS_S4 in predicting SRI-6 ≤ −0.5 are generally quite high (around 0.8), but some lower values such as 0.72 (this for the station with largest contributing area) are observed (Fig. 5).The lower skills for the station with the largest contributing area for FS_S4 might be attributed  to the shift from an arid to a more tropical climate, which means that the persistence of initial conditions would be lower.Also, given that this is mostly the case for the FS_S4 and less so for the FS_ESP and FS_ESPcond, we can speculate the ECMWF S4 seasonal forecast might have a better skill for the northern (more arid) part of the basin (area corresponding to the subbasin draining to Station 1) than for the southern part of the basin.FS_ESP generally shows the lowest skills, with the skills of FS_ESPcond in FS_ESP and FS_S4.The verification was also done for forecasts issued for the 4-month period JFMA (high-runoff season) with forecasts issued from November to January, respectively.Figure 6 presents the ROCS for the 4-monthly SRI (SRI-4 in April) ≤ −0.5 for three different lead times (3-5 months) and two stations.
For the high runoff season SRI-4, similar results to those of SRI-6 are observed.In almost every case FS_S4 shows higher skill than FS_ESP and FS_ESPcond.The skill of the forecasts tend to decrease with lead time, for FS_ESP and FS_ESPcond, which do not show any skill at the 5-month lead time.In contrast, the skill of FS_S4 for the 5-month lead time is still good.The skill score verification for SRI-1 for the same 4-month period January-April (not shown) shows once more that the FS_S4 is more skilful than the other two forecasting systems.The smaller subbasins (18 and 20) present lower skill for SRI-4 for the three forecasting systems, and while subbasin 18 still presents some skill for all the three FS, subbasin 20 only does so for the FS_S4 for all lead times.In general, for all locations, the skill of the FS_S4 decreases slightly with lead time, while the skill of both FS_ESP and FS_ESPcond decreases more rapidly with lead time.A curious fact is that for a few stations (e.g. 1, 18), for both SRI-4 and SRI-1, the FS_S4 shows a higher skill for a lead time of 4 months than for a lead time of 3 months.For the SRI-4, this means that the forecast is more skilful in predicting the April SRI-4 when issued in December than when issued in January.However, the differences are not statistically significant and this can be due to sampling errors.
Rank histograms for every station and lead time together with the results of the Kolmogorov-Smirnov test show that for the three forecasting systems, uniformity of the distribution cannot be rejected, indicating the forecasts are reliable.Figure 7 presents the rank histograms of SRI-6 for Station 1 for the three forecasting systems as an example.

Skill of spatial hydrological indicators
Figure 8 shows the ROCS and the BSS of the FS_S4 in predicting agricultural drought conditions, i.e. in predicting aggregated RS during the 6-monthly period DJFMAM to be higher than the 70th percentile.Yuan et al. (2013) show that the annual cycle of soil moisture in southern Africa (simulated by the VIC model) lags behind the precipitation.Figure 8 shows that the skill of the FS_S4 forecast in predicting agricultural droughts is higher than climatology (ROCS > 0.5, BSS > 0) throughout the entire basin.
To assess the skill of the seasonal forecast in predicting a specific decision variable in the Limpopo River basin, we calculate the skill of the forecast in predicting water level thresholds in the reservoir that would result in curtailment to the irrigation sector.The availability of water is represented in each cell by the water level.In the cells corresponding to reservoirs, the water level is a surrogate for the storage and is described as a percentage of the full storage capacity of the reservoir.Figure 9 presents the ROCS and the BSS of the FS_S4 in predicting water levels during the 6-monthly period DJFMAM to be lower than the 50th and 37.5th percentiles, based on the analysis described in Sect.2.4.The figure shows that the skill of the FS_S4 forecast in predicting low water levels is higher than climatology (ROCS > 0.5, BSS > 0) throughout the basin.The spatial distribution across the basin does show the skill to be higher in the northern basin than in the southern basin, which may contribute to the lower skill found at Station 24 close to the basin outlet than at Station 1 in the upper (northern) basin.
The skill scores in cells that contain the reservoirs are represented by a circle to enhance visibility.It is clear from the figure that the skill of the forecast in predicting low water levels is higher in the reservoirs than in nearby streams.This can of course be expected due to the higher memory introduced by the reservoir's storage capacity with respect to the streams.Figure 10 presents the forecast probability of water levels to be lower than the 50th and 37.5th percentiles during the December 1991-May 1992 season as an example.This was the driest season in the last 30 years.The forecast is issued in December 1991.
The forecast probability of water levels in the reservoirs being lower than the 50th and 37.5th percentiles can be interpreted as the forecast probability of a curtailment of 20 and 65 %, respectively, in the irrigation sector during the season.For several reservoirs in the basin the FS_S4 forecast issued in December 1991 predicted a high probability of curtailment to the irrigation sector during the December 1991-May 1992 season.Records confirm the lower than normal water levels during this season, with the irrigation quota indeed being curtailed (DWA, 2013).Yuan et al. (2013) note that "The major source of seasonal forecast predictability comes from the ocean, and the strongest signal is the El Niño Southern Oscillation (ENSO)".Given that the ECMWF S4 is influenced by the ENSO signal, it is interesting to analyse how the FS_S4 predicts streamflow in the onset of two clear El Niño years.The 1997/98 El Niño year is described in Thomson et al. (2003) as the largest for this century, predicted with a high degree of certainty.Although many of the climate anomalies typical of an El Niño event took place around the globe, the devastating drought that was feared for southern Africa did not happen (Thomson et al., 2003).For this analysis another year was selected that had a less strong ONI but that did result in a severe drought (1982/83).Figure 11 presents the ensemble seasonal streamflow prediction from FS_S4 for both the 1997/98 and the 1982/83 seasons issued in October and updated in December for Station 24.The plots also show the climatology of the streamflow and the 30th percentile, i.e. the value below which 30 % of the observations are found.The reference streamflow for that season and the forecast ensemble mean are also shown.

Analysis of a specific event
Figure 11 shows that in October the predictions from the forecasting system FS_S4 for El Niño seasons of 1982/83 and 1997/98 were relatively similar (see Fig. 11 upper pan-  els) even though the 1997 JAS ONI was notably higher than the 1982 JAS ONI.The updated forecast in December, however, shows a different situation: while the forecast for the 1982/83 season point towards very dry conditions, the forecast of the 1997/98 season indicates near-normal conditions.Yet, the 1997 SON ONI is markedly higher than the 1982 SON ONI.Thus, in spite of the strong ONI conditions, the S4 system correctly forecasted the no-drought condition in the 1997/98 season.This indicates that even though the S4 forecasting system is influenced by the sea surface temperatures over the Niño-3.4region, the precipitation and temperature forecasts over the Limpopo region are not only constrained by the sea surface temperature evolution but results from the atmospheric circulation response to different climate forcing.

Discussion
The performance of the three hydrological forecasting systems constructed with the same hydrological model and different meteorological ensemble forecasts are evaluated by means of widely used probabilistic verification skill scores, including the ROC diagram and the rank histogram.Among the forecasting systems considered in this study, FS_ESP is considered the most traditional.Such traditional approaches for hydrological forecasts rely on historical observations of the meteorological conditions, without considering meteoro-logical forecasts.In ensemble probabilistic forecasting, the ESP approach, implicitly accounting for hydrologic persistence and historical variability of climate, is normally used (Brown et al., 2010).FS_S4 is a more complex forecasting system as it requires as forcing the outputs of a seasonal meteorological forecast system, which are complex numerical models and resource intensive.FS_ESPcond, a modification of the ESP approach, conditions its ensemble on past years that had similar climate conditions to the year in which the forecast is made (Brown et al., 2010).Given that the Limpopo region is known to be affected by ENSO and droughts tend to occur during El Niño years, the forecast ensemble was constructed by assigning weights to the different ensemble traces based on the El Niño index.
The skill evaluation of the seasonal forecasts is limited by the use of model data as verification, i.e. we verify our forecasts against the baseline simulation, which was also used to provide the initial conditions to the forecasts.This is the same approach as taken in Yossef et al. (2013), Winsemius et al. (2014), Shukla et al. (2013) and Renner et al. (2009), and while it allows for the detailed (spatial) evaluation of the skill of the forecasts, it can potentially hide limitations of the modelling system.Therefore, these skill results should be interpreted as the upper limit of real predictability of the current system.Results of the seasonal streamflow prediction show that for every lead time FS_S4 is skilful in predicting SRI-6, SRI-4, and SRI-1 during the summer rainy season,  while for the other FS the skill is lower and decreases more rapidly lead time.This means that the complex S4 seasonal forecasting system adds value to the hydrological predictions compared to the climatology-based forecasting systems, as well as the ENSO-mode-conditioned climatology forecast systems.This was also observed during a specific event where expected anomalies due to El Niño did not materialise, but FS_S4 detected this.The skill decreases when going from SRI-6 to SRI-4 and SRI-1.This is as expected given the higher variability of the predictand for shorter aggregation periods.The skill from FS_ESP is lower than that of FS_S4 in almost every case, while the skill of FS_ESPcond is in general between the other two.For SRI-4, FS_ESP and to a lesser extent FS_ESPcond do not show any skill for a 5-month lead time at any of the stations considered.
As expected, the skill of all forecasts tends to decrease with lead time.This is, however, especially the case for FS_ESP and FS_ESPcond where the decrease in skill with lead time is larger than for FS_S4.For the smaller aggregation periods (SRI-4 and SRI-1) FS_ESP deteriorates to climatology already at a lead time of 3 months for stations 18 and 20, the upstream basins of which are smaller in size.In the larger basins FS_ESP shows predictability up to a 4-month lead time, probably due to the spatial aggregation taking place over larger basins smoothing out uncertainties in space.This indicates that the memory in the hydrology (storage in groundwater, reservoirs, channels and wetlands) contributes to the predictability with a lead time of up to 2-4 months.For longer lead times, the meteorological forcing dominates the predictability of the system.The critical lead time after which the importance of the meteorological forecast exceeds that of the initial conditions depends on the location and size of the basin and should be analysed for each subbasin of interest.Rank histograms for every station and lead time indicate that the three forecast systems are reliable given that uniformity of the distribution cannot be rejected.

What does the analysis mean to end users?
The high predictability of FS_S4 for all lead times and aggregation periods of SRI is encouraging given that such a system, if made operational, may provide end users with sufficient time to decide upon measures to take in anticipation.For example, they might decide to change the cropping date or the cropping area if they expect not to have enough water to fulfil the crop requirements.Therefore, there is added value to using a seasonal meteorological forecast (ECMWF S4) to force the hydrological forecasting system when compared to the conventional ESP.The higher skill of the FS_S4 and FS_ESPcond compared to that of the FS_ESP for every lead time is in line with the study of Shukla et al. (2013), who show that for the region of the Limpopo River basin the meteorological forecast dominates the hydrological predictability for the wet season for almost every lead time considered.Only for the 1-month lead time forecasts issued in October did they find a higher influence of the hydrological initial conditions to some extent.Moreover, Yossef et al. (2013) indicate that for semi-arid regions initial conditions do not contribute much to the skill given the high sensitivity of the runoff coefficient to rainfall variability.
The FS_S4 was also evaluated regarding its ability to predict agricultural droughts and curtailments in irrigation (water levels lower than the 50th and 37.5th percentiles).Maps of spatially distributed ROCS and BSS (Figs. 8,9) show that the skill of the FS_S4 forecast in predicting these conditions is higher than climatology (ROCS > 0.5, BSS > 0) throughout the basin.Indicating the probability of curtailment to the irrigation sector during the following season is an example of providing a forecast in an understandable format that is useful to the end users.If they are informed that there is a high probability of a high curtailment to the water available for irrigating their crops during the following season, users would have a clear idea of what is the best practice for that situation.Further improvements in forecasting skill could be achieved through better meteorological predictions or better estimation of initial conditions (Yossef et al., 2013).Whether the forecasts indeed have value will depend on the costs of decisions made in response to the forecast, losses in case of a wrong decision and the gain in case of a good decision.This should be further analysed in a continuation of this study.
As a next step, it is recommended that the forecast skill of the FS_S4 and FS_ESPcond be assessed in an actual forecasting mode for a following summer season.The seasonal meteorological forecast from S4 can be obtained in real time for research purposes.To test a pre-operational system, the forecasting system ought to be statistically post-processed in order to remove biases in streamflow predictions.Moreover, the initial conditions for the forecasts could be better estimated through data assimilation of water levels in reservoirs and streams.This data could be obtained from the water managers of the basin.Despite the limitations of FS_S4 (access to real-time atmospheric-ocean seasonal forecasts for non-ECMWF member-states, and their quality) and FS_ESPcond (depending on the calibration and decreased skill at long lead times), both systems show potential for seasonal hydrological drought forecasting in the Limpopo River basin to provide operational guidance to users.

Conclusions
We evaluate the performance of three forecasting systems (FS_S4, FS_ESP, and FS_ESPcond) in the Limpopo River basin.These systems make use of the same hydrological model and are forced with three different meteorological ensemble forecasts (two of which are based on resampled climatological records, FS_ESP and FS_ESPcond, and one based on seasonal meteorological forecasts, FS_S4).Results of the seasonal streamflow prediction show that the three forecasting systems show moderate skill in predicting SRI-6 (DJFMAM) ≤ −0.5.Moreover, the three forecasting systems are unbiased as suggested by the rank histograms.
For every lead time and aggregation period considered, FS_S4 is found to be skilful in predicting hydrological droughts represented by SRI≤ −0.5 during the summer rainy season.The skill decreases when going from SRI-6 to SRI-4 and SRI-1, as well as with increasing lead time.The skill of FS_ESP is lower than that of FS_S4 in almost every case and deteriorates rapidly with lead time, showing no skill af-ter a lead time of 4-5 months for SRI-4 and SRI-1.This indicates that the memory in the hydrology contributes to the predictability up to 2-4 months but for longer lead times the predictability of the system is dominated by the meteorological forcing.FS_ESPcond shows in general lower than FS_S4 but it becomes comparable and can even outperform the latter for smaller lead times if the parameters for selection and weighting of ensemble members are carefully calibrated for each basin.Moreover, the skill of FS_ESPcond is more robust than that of the other forecasting systems as suggested by the narrower confidence intervals of ROCS.As with FS_ESP, the skill of FS_ESPcond also decreases faster than that of FS_S4 with lead time.
The high predictability of drought of FS_S4 for all lead times and aggregation periods of SRI and for the spatial drought indicators is encouraging given that such a system, if made operational, may provide end users with sufficient time to decide upon measures to take in anticipation.Moreover, FS_ESPcond shows promising results.This forecasting system only requires the ONI index previous to the forecast to weight the ensemble traces to include in the forecast.This system is relatively simple and presents the advantage that it can be coupled with the forecast of the ONI index that is available with a long lead time.Naturally, in this situation the uncertainties of both forecasts need to be considered.

Figure 2 .
Figure 2. Location of the Limpopo River basin.

Figure 3 .
Figure 3. Locations of selected hydrometric stations and reservoirs in the Limpopo Basin.

Figure 4 .
Figure 4. Upper plot: Limpopo River flow regime for Station 24 at Chókwe.The blue line represents the average observed runoff, and the whiskers of the boxplots represent the 10th percentile and the 90th percentile.The lighter and darker shaded areas represent the main runoff period and high runoff period, respectively.Lower plot: initialization dates and length of forecasts during the year.The forecast issued in December is highlighted as the one that captures the main runoff season.

Figure 5 .
Figure 5. Upper plots: relative operating characteristic (ROC) diagram representing false alarm rate versus hit rate for the 6-month SRI (DJFMAM) ≤ −0.5 given by FS_S4, FS_ESP and FS_ESPcond for three stations (1, 20,and 24).The ROCS for each forecasting system together with the 90 % confidence interval (5-95th percentiles) resulting from the bootstrap are indicated in the legend.Middle and bottom plots: histogram of the bootstrapped ROCS for FS_S4 (middle) and FS_ESPcond (lower), respectively, for the same three stations.

Figure 7 .
Figure 7. Rank histograms of SRI-6 for Station 1 for the three forecasting systems FS_ESP, and FS_ESPcond).The results of the Kolmogorov-Smirnov test for uniformity are presented in each plot.

Figure 11 .
Figure 11.Seasonal forecast FS_S4 for 2 seasons issued in October (upper panel) and December (lower panel).

Limpopo River basin 2.1.1 Study area -Limpopo River basin
with the ERA-Interim forcing meteorological data at a daily time step.ERA-Interim (ERAI) is the latest global atmospheric reanalysis produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) and covers the period from January 1979 to the present date with a horizontal resolution of approximately Hydrol.Earth Syst.Sci., 19, 1695-1711, 2015 www.hydrol-earth-syst-sci.net/19/1695/2015/ 0.7 degrees and 62 vertical levels.A complete description of the ERAI product is available in

Table 1 .
Model evaluation measures for runoff for selected stations, ordered by basin size.