Ensemble hydrological predictions are normally obtained by forcing hydrological models with ensembles of atmospheric forecasts produced by numerical weather prediction models. To be of practical value to water users, such forecasts should not only be sufficiently skilful, they should also provide information that is relevant to the decisions end users make. The semi-arid Limpopo Basin in southern Africa has experienced severe droughts in the past, resulting in crop failure, economic losses and the need for humanitarian aid. In this paper we address the seasonal prediction of hydrological drought in the Limpopo River basin by testing three proposed forecasting systems (FS) that can provide operational guidance to reservoir operators and water managers at the seasonal timescale. All three FS include a distributed hydrological model of the basin, which is forced with either (i) a global atmospheric model forecast (ECMWF seasonal forecast system – S4), (ii) the commonly applied ensemble streamflow prediction approach (ESP) using resampled historical data, or (iii) a conditional ESP approach (ESPcond) that is conditional on the ENSO (El Niño–Southern Oscillation) signal. We determine the skill of the three systems in predicting streamflow and commonly used drought indices. We also assess the skill in predicting indicators that are meaningful to local end users in the basin. FS_S4 shows moderate skill for all lead times (3, 4, and 5 months) and aggregation periods. FS_ESP also performs better than climatology for the shorter lead times, but with lower skill than FS_S4. FS_ESPcond shows intermediate skill compared to the other two FS, though its skill is shown to be more robust. The skill of FS_ESP and FS_ESPcond is found to decrease rapidly with increasing lead time when compared to FS_S4. The results show that both FS_S4 and FS_ESPcond have good potential for seasonal hydrological drought forecasting in the Limpopo River basin, which is encouraging in the context of providing better operational guidance to water users.
Climate change studies show evidence of an intensification of the global water cycle (Huntington, 2006; IPCC, 2007; Hansen et al., 2012; Trenberth, 2012; Coumou and Rahmstorf, 2012), with extreme events including floods and droughts expected to become more frequent. The UNISDR (United Nations Office for Disaster Risk Reduction) Hyogo Framework of Action 2005–2015 (UNISDR, 2005) describes early warning systems and action plans triggered on the issuing of a warning as one of the most effective strategies to mitigate the impacts of natural hazards. Operational forecasting of streamflow to inform early warning is already commonplace in several parts of the world, but the main focus is often on flood prediction. Operational forecasting of streamflow for drought prediction has to date not been applied as widely, despite the widespread recognition of the relevance and importance of drought forecasting in the research community.
There are several Drought Early Warning Systems (DEWS) currently in existence in the world, though due to the complexity of drought these are arguably less developed than many flood early warning systems. Grasso (2009) reports that only three institutions provide information on the occurrence of major droughts at the global scale: FAO's Global Information and Early Warning System on Food and Agriculture (GIEWS), the Humanitarian Early Warning Service (HEWS) operated by the World Food Programme (WFP), and the Benfield Hazard Research Centre at University College London.
In the United States the US Drought Monitor
(
In Europe the European Commission Joint Research Centre (JRC) has
established the European Drought Observatory (EDO,
The FEWS Net (Famine Early Warning Systems Network) for eastern Africa, Afghanistan, and Central America reports on current famine conditions, including droughts, by providing monthly bulletins that are accessible on the FEWS Net web page. However, a drought forecast is not provided. Other drought warning systems over Africa include the Botswana national early warning system (EWS) for drought (Morgan, 1985) and the Regional Integrated Multi-Hazard Early Warning System for Africa and Asia (RIMES). In the latter a drought early warning system is being adapted to identify climate and water supply trends in order to detect the probability and potential severity of drought (RIMES, 2014).
Advances regarding drought early warning systems in Africa in the last few
years are remarkable. There is an increasing availability of drought
monitoring and forecasting tools for decision making that can provide real-time monitoring and forecasting of drought across the continent. The Land
Surface Hydrology Group at Princeton University, USA, has recently
established an African Flood and Drought Monitor
(
Yuan et al. (2013) applied the NCEP's Climate Forecast System version 2 (CFSv2) combined with the VIC land surface model for seasonal drought prediction over Africa. They used both the SPI and soil moisture as indices and the Brier skill score (BSS) to assess the probabilistic drought hindcast for 1982–2002. Their results show relatively good skill in the dry season but only limited skill in the rainy season. They indicate that CFSv2 precipitation is correlated with the observed precipitation over southern Africa, but only accounts for 44–45 % of the variance of observations. They point out that for two extreme droughts CFSv2 predicted neutral conditions or only a weak anomaly. Our study focuses on the Limpopo River basin and follows a similar type of analysis, although it does so at a higher resolution and in a more detailed manner. Additionally, we present different skill scores for different hydrological drought indicators during the rainy season, and compare different forecasting systems in the basin. We focus on assessing the skill of the forecast in predicting indicators that are meaningful to the local end users in the basin.
The semi-arid Limpopo River basin, located in southern Africa, has experienced severe droughts in the past, which have led to crop failures, high economic losses and the need for humanitarian aid. An effective drought early warning system for this basin is of prime importance. Current practices for drought forecasting in the Limpopo River basin involve three forms of seasonal climate forecasts ranging from regional to local scales: the Southern Africa Regional Climate Outlook Forum (SARCOF) climate outlooks, seasonal climate outlooks prepared by meteorological departments, and forecasts based on local knowledge applied in rural communities. Despite these seasonal forecasts being available in the basin, farmers seem to prefer to rely on drought forecasting systems based on indigenous and traditional knowledge. Such forecasts include signs in (i) the sun, moon and wind; (ii) trees and plants; and (iii) insects, birds and animals (DEWFORA, 2013). For seasonal forecasts to be accepted by the local community there are several challenges that need to be addressed. End users should receive the information in a suitably understandable format at the time they need it for the forecast to be useful. The highly technical information that is typically contained in the forecasts should then be translated to a comprehensible form before being disseminated and delivered to decision makers and farmers. Moreover, end users should be involved in the product verification by providing feedback to the forecasters (DEWFORA, 2012).
Seasonal hydrological drought forecasts aim for high hydrological predictability at a seasonal timescale. Shukla et al. (2013) quantified the contribution of a good representation of initial hydrologic conditions (IHCs) and seasonal meteorological forecast (MF) to seasonal hydrological predictability at different forecast dates and lead times (1, 3, and 6 months) globally. They quantified the contributions of two components of the IHCs (soil moisture and snow water content) through ensemble streamflow prediction (ESP) and reverse-ESP. Their results show that for the region of the Limpopo River basin the MF dominates the hydrological predictability during the wet season (forecasts starting in October and January) for almost every lead time considered. Only for the 1-month lead time forecasts issued in October did the IHCs appear to some extent to have a higher influence. For the dry season the IHCs dominate the hydrological forecast at all lead times. These results suggest that to improve the seasonal hydrologic forecast skill in the Limpopo River basin, efforts should focus on improving the MF. However, the contribution of other IHCs (surface water and groundwater level) to hydrological predictability should also be assessed.
Yossef et al. (2013) also investigated the relative contribution of initial conditions and meteorological forcing to the skill of the global seasonal streamflow forecasting system FEWS-World, using the global hydrological model PCR-GLOBWB (PCRaster Global Water Balance). They use ESP and reverse-ESP to determine the critical lead time for different locations at which the importance of the initial conditions is surpassed by that of the meteorological forcing. They indicate that for semi-arid regions such as the Limpopo Basin the initial conditions do not contribute much to the skill given the high sensitivity of the runoff coefficient to rainfall variability. This would suggest that the predictability in semi-arid basins such as the Limpopo using ESP is limited, with seasonal meteorological forecasts potentially offering better skill.
In this study we introduce three dynamic forecasting systems based on a distributed hydrological model for the seasonal prediction of hydrological droughts for the semi-arid Limpopo Basin in southern Africa. All three forecasting systems include a distributed hydrological model of the basin, and are forced by either (i) a global atmospheric model (ECMWF seasonal forecast system S4), (ii) the ESP approach using resampled historical data, or (iii) a conditional ESP approach (ESPcond) that is conditioned on the ENSO (El Niño–Southern Oscillation) signal. The aim of this study is to assess the skill of the three systems in predicting meaningful drought indices for the Limpopo Basin.
The approach followed in this study is summarised in Fig. 1. It starts with obtaining the meteorological seasonal forecast and preprocessing the data. This is then used to force the hydrological model (embedded in the Delft-FEWS forecasting shell; Werner et al., 2013), thus obtaining seasonal forecasts of streamflow and other hydrological variables.
Approach followed in the forecasting system for the Limpopo River basin.
The Limpopo River basin is one of the larger basins in southern Africa, with
a drainage area of approximately 415 000 km
Location of the Limpopo River basin.
The annual rainfall in the basin averages some 530 mm yr
Locations of selected hydrometric stations and reservoirs in the Limpopo Basin.
A finer-resolution version (0.05
The hydrological model is embedded in the Delft-FEWS (Flood Early Warning System) open shell for forecasting purposes. The shell provides a sophisticated collection of modules designed for building a hydrological forecasting system customised to the specific requirements of an individual organisation. The philosophy is to provide an open shell for managing the data handling and forecasting process. This shell incorporates a comprehensive library of general data handling utilities, allowing a wide range of external models to be integrated in the system through a published open interface. This allows existing simulation models and data streams to be incorporated into a comprehensive and reliable forecasting system (Werner et al., 2013).
The hydrological model is run in simulation mode for a 32-year period
(1979–2010) with the ERA-Interim forcing meteorological data at a daily time
step. ERA-Interim (ERAI) is the latest global atmospheric reanalysis
produced by the European Centre for Medium-Range Weather Forecasts (ECMWF)
and covers the period from January 1979 to the present date with a
horizontal resolution of approximately 0.7 degrees and 62 vertical levels. A
complete description of the ERAI product is available in Dee et al. (2011). The ERA-Interim precipitation data used in this study was
corrected using the GPCP (Global Precipitation Climatology Project) v2.1
product
to reduce the bias when compared to measured products
(Balsamo et al., 2010). The GPCP v2.1 data is a monthly
climatology provided globally at 2.5
In addition to the precipitation, other meteorological parameters from the
ERA-Interim reanalysis data that are used to force the model include the 2 m daily temperature (minimum, maximum and average). Temperature data is
used for the computation of the reference potential evaporation that is
required to force the hydrological model. In this study the Hargreaves
formula is used. This method requires less parameterization than the
Penman–Monteith formula, though it has the disadvantage that it is less
sensitive to (uncertain) climatic input data, with a possible reduction of
the dynamics and accuracy of the potential evaporation as a consequence.
However, this also means that it is less sensitive to errors in climatic
inputs (Hargreaves and Allen, 2003) that are inherent to any
meteorological forecast. Moreover, the choice of the method used for the
computation of the reference potential evaporation was shown to have minor
effects on the results of the actual evaporation for southern Africa, where
actual evaporation is dominated by soil moisture availability
(Trambauer et al., 2014a). The ERA-Interim data for
the 32-year period from 1979 to 2010, corrected using the GPCP v2.1 data set,
are converted to the same spatial resolution as the continental-scale
version of the PCR-GLOBWB model. ERAI is archived on an irregular grid
(reduced Gaussian) with an approximate resolution of 0.7
The reference run provides the initial conditions for all forecasts. Initial conditions at the beginning of each month are saved in the Delft-FEWS database, and subsequently used as “warm states” to start the forecasts when doing the retroactive forecast (also referred to as hindcasts).
An ensemble of meteorological hindcasts is first tested for the summer rainfall season over southern Africa for the period 1981–2010. Seasonal forecasts in this study are issued for only 7 months of the year so as to capture the rainy season and main runoff season (meaning the there are 5 months where we do not issue a forecast). The predictive skill for drought is expected to be higher during the dry season and lower during the wet season given that the hydro-climate has a longer persistence during the dry season (Yuan et al., 2013). Yuan et al. (2013) show the high contrast in skill between the dry and wet seasons in southern Africa.
In the hindcast, the first forecast of each season is issued in August and includes the seasonal (6 months) forecast from August to January. The forecast is updated at the beginning of each month from September to February. The last forecast of the season is issued in February, covering the period from February to July (see Fig. 4). All simulations are done at a daily time step.
Upper plot: Limpopo River flow regime for Station 24 at Chókwe. The blue line represents the average observed runoff, and the whiskers of the boxplots represent the 10th percentile and the 90th percentile. The lighter and darker shaded areas represent the main runoff period and high runoff period, respectively. Lower plot: initialization dates and length of forecasts during the year. The forecast issued in December is highlighted as the one that captures the main runoff season.
All three forecasting systems considered use the same hydrological model of the basin, but are forced with different meteorological forecasts. In the first system (FS_S4) the PCR-GLOBWB hydrological model is forced with the output of a global atmospheric model, the ECMWF seasonal forecast system S4 (atmosphere–ocean coupled). The second forecasting system (FS_ESP) is based on the ESP (Day, 1985) procedure. In the ESP procedure the ensemble meteorological forecast is generated with re-sampled historical meteorological data. The hydrological model is then forced with this re-sampled data. A third system (FS_ESPcond) is proposed given that the ENSO has a clear influence on the interannual climate variability over the Limpopo River basin (Landman and Mason, 1999). This is equivalent to the second system but the weights of the ESP ensemble members are conditioned on the ENSO signal (Oceanic Niño Index, ONI). This is explained in full in Sect. 2.2.3.
Seasonal meteorological forecasts from the most recent seasonal forecasting
system at ECMWF (S4) are used to force the hydrological model. The S4
ensemble seasonal forecasts are initialised on the 1st of each month
and the ensemble is generated by perturbations in the initial conditions and
by the use of stochastic physics in the atmosphere during the model
integration (out to 6 months lead time; Molteni et al.,
2011). The atmospheric resolution is about 79 km with 91 vertical levels,
and is fully coupled with an ocean model with a horizontal resolution of
1
Mean biases and drifts in the seasonal forecasts of precipitation can have a
detrimental influence on the hydrological forecasts. Therefore, a simple
climatological bias correction, based on monthly means, is applied to the
seasonal forecasts in the form
A widely used approach to seasonal forecasting is the ESP procedure. ESP predicts future streamflow from the current initial conditions (warm state) in the hydrological model with re-sampled historical meteorological data (ERA-Interim-corrected with GPCP-observed meteorology from the last 31 years in this study). The procedure assumes that meteorological events that occurred in the past are representative of events that may occur in the future (Day, 1985). Although ESP is normally used in the absence of a seasonal forecast, in this study we use it to compare the skill of the FS_ESP with that of the FS_S4. Moreover, a comparison of these two forecasts may give an indication of what influences the predictability. ESP represents forecast uncertainty due to boundary forcing uncertainties only (Wood and Lettenmaier, 2008) and thus allows measuring the skill that can be expected only from initial states. In the FS_ESP hindcast, the sample of the year in which the forecast starts is excluded from the ensemble to allow for a fair estimate of the forecast uncertainty. The FS_ESP therefore includes 30 (31 minus 1) years in the ensemble.
ENSO is clearly related to interannual climate variability over the Limpopo River basin. In southern Africa meteorological droughts tend to happen in the December–March rainy season after onset of an El Niño event (Thomson et al., 2003). However, it is not always the case that this happens. Thomson et al. (2003) recorded a 120 % increase in probability of drought disaster in the year after an El Niño onset. To account for the relationship between ENSO and the occurrence of drought, this system is similar to FS_ESP but the weights of ensemble members sampled through the ESP procedure are conditioned on the ENSO signal.
We use the post-ESP weighting technique described in Werner et al. (2004). This approach uses the El Niño-3.4 index averaged over the 3-month-period immediately prior to the issue date of the forecast to weight ensemble members from ESP. The technique is summarised here for the forecast of the 6-month standardised runoff index (SRI-6).
Compute a vector ( Compute a vector of weights ( Calculate the probability (
The parameters
For the FS_ESPcond we chose to keep the parameters constant
(
We also use this procedure for the forecast of SRI-4 (JFMA SRI). FS_ESPcond always uses the latest ONI index available prior to the start date of the forecast. This means that for the forecast issued in January, which corresponds to a 3-month lead time, FS_ESPcond uses the ONI values for October, November and December (OND). Similarly, for the forecast issued in December, which corresponds to a 4-month lead time, FS_ESPcond uses the SON (September, October, November) ONI, and the forecast issued in November (5-month lead time) makes use of the ASO (August, September, October) ONI.
Standard verification skill scores are selected to measure the skill of the forecast ensembles in predicting drought indicators. In this study we use the SRI for the characterisation of hydrological droughts. This indicator is explained in the following section. Forecasts are verified against the reference run and the resulting skills are established relative to sample climatology. Cloke and Pappenberger (2008) recommend the use of several verification measures in the same analysis so that the quality of the forecast can be assessed rigorously. We selected three verification scores that measure slightly different properties of the forecast skill. The ROC curve measures discrimination but not bias, the rank histogram measures reliability or bias, and the Brier score (BS) accounts both for reliability and sharpness (Renner et al., 2009).
The ROC (relative operating characteristic, or receiver operating characteristic) diagram measures the ability of the forecast to discriminate between two alternative outcomes. It plots the hit rate or probability of detection (POD) versus the false alarm rate or probability of false detection (POFD). It is not sensitive to bias in the forecast, so it says nothing about the reliability. It is conditioned to the observations. In summary, it indicates the ability of the forecast to discriminate between events and non-events given a certain event threshold (WWRP/WGNE, 2013). The area under a ROC curve (ROCS) is used as a score. ROCS can take values from 0 to 1, with a value of 0.5 indicating no skill and a value of 1 representing a perfect score. Values lower than 0.5 indicate negative skill. ROC curves measure how good forecasts are in the context of a very simple decision-making model, and are thus better suited to measure how good forecasts are from the perspective of the user than many other commonly used measures (Tveito et al., 2008).
The BS ([0–1]) measures the mean squared probability error
and represents the magnitude of the probability forecast errors, with a
perfect score of zero. The BSS ([
The rank histogram is used to evaluate whether the forecast ensembles are from the same underlying population as the observations, which implies that the observed would have the same probability of occurrence as any of the ensemble members. This would result in a uniform distribution in the histogram that plots the frequency of the rank of the observation in the ensemble, while deviations from the uniform distribution reveal deficiencies in ensemble calibration or reliability (Wilks, 2011).
The hydrological drought indicator SRI follows the same concept as the SPI and is defined as a “unit standard normal deviate associated with the percentile of hydrologic runoff accumulated over a specific duration” (Shukla and Wood, 2008). To compute SRI the runoff time series is fitted to a probability density function (a gamma distribution) and the function is used to estimate the cumulative probability of the runoff of interest for a specific month and temporal scale. The cumulative probability is then transformed to the standardised normal distribution with mean of 0 and variance of 1 (Shukla and Wood, 2008).
Forecasted streamflow is transformed to the hydrological drought indicator
SRI and forecasts of drought are analysed by considering drought conditions
to occur for SRI
However, as we also want to analyse the ability of the system to forecast distributed variables (for agricultural droughts) and water levels in the reservoirs (for irrigation curtailments), we also evaluated the skill of the forecast system in predicting these variables.
Given the small sample size resulting from applying the verification over the 30-year hindcast period, a bootstrap approach is used to estimate the confidence intervals around the ROCS. The idea behind the bootstrap is to treat a finite sample at hand as similarly as possible to the unknown distribution from which it was drawn, which in practice leads to resampling with replacement (Wilks, 2011). The uncertainty of the ROCS is estimated by applying a bootstrap resampling with replacement procedure.
For the FS_S4 and FS_ESP forecasts, we randomly replace (allowing repetition) the original forecast and verification pair to produce a new sample of the same size as our original sample. We then calculate the ROCS from the new sample. We repeat this procedure to create 1000 new samples from which we generate an empirical distribution of the ROCS. The 90 % confidence interval is estimated from the 5th and 95th percentiles of this empirical distribution.
For the FS_ESPcond the bootstrap procedure follows the same theory but is computed slightly differently. In this case the bootstrap is achieved by recreating the ensemble forecasts for the hindcast period 1000 times based on the computed probability vector and computing the skill score from each created ensemble.
A limitation of this bootstrap procedure is that statistics computed from discrete bootstrap samples may differ from the ones based on continuous data, and this might lead to overestimation of the confidence. However, this method is widely used in the literature (Dutra et al., 2014; Friederichs and Thorarinsdottir, 2012; Wilks, 2011) to estimate confidence intervals as it does not require assumptions on the distribution.
ROCS and BSS are computed for the spatially distributed indicator root stress (RS) to assess the skill of the forecast in predicting agricultural drought indicators. The RS is an indicator of the available (or the lack of) soil moisture in the root zone, which can be calculated for each grid cell. The RS varies from 0 to 1, where 0 indicates that the soil water availability in the root zone is at field capacity and 1 indicates that the soil water availability in the root zone is at wilting point and the plant is under maximum water stress. For each grid cell, a drought is defined to occur when the root stress is higher than the 70th percentile of the observed values for that month. An advantage of defining the threshold as a percentile of the observed sample as proposed by Roulin (2007) is that it assures a sufficiently large enough number of events to verify and also allows for comparison of verification statistics at different locations (Renner et al., 2009).
In addition to indicators such as RS, it is interesting to evaluate the skill of the model in predicting indicators that are meaningful to the end users in the basin. Irrigation is the major water use in the Limpopo Basin. The amount of water made available to the irrigation sector may, however, be restricted depending on the water level in the reservoirs in the basin as a percentage of their full capacity (DWA, 2013). The forecasted anomaly of the water level in the reservoir is a decision variable that can give an indication to the water managers of the percentage of irrigation demand that can be covered during the season.
An analysis of the historical time series of water level for the Tzaneen reservoir together with the curtailment rules of the reservoir (DWA, 2013) indicate that a 20 % curtailment to the irrigation sector is applied when water levels in the reservoir fall below the 50th percentile in the water levels (in percentage of the capacity of the reservoir). Similarly, a 65 % curtailment to the irrigation sector is applied when water levels in the reservoir fall below the 37.5th percentile and a 90 % curtailment in the irrigation sector along with a 30 % curtailment in the urban sector when the water levels are below the 12th percentile. ROCS and BSS are then computed to assess the skill of the forecast in predicting the water levels in the reservoirs to be lower than these threshold percentages of the full capacity. Although the actual operation of the reservoirs is quite a bit more complex, this can be interpreted as an assessment of the skill of the forecast in predicting curtailments to the irrigation sector.
The following section outlines the results when applying the different types of forcing to the hydrological model over the 30-year hindcast period from 1981 to 2010. The analysis is carried out for different verification periods and lead times as the forecast quality may vary significantly with temporal scales and lead times. While the rainy season in the Limpopo River basins spans from October to March, the main rains typically take place from November to February. The main runoff season and the high runoff season, however, lag behind the rainy season by 1 or 2 months, occurring in general from December to May and from January to April respectively (see Fig. 4).
This section presents the skill expressed in the selected skill scores of
the seasonal streamflow prediction for the three forecast systems described
(FS_S4, FS_ESP, and FS_ESPcond)
for Station 24 (Chókwe), Station 1, Station 18 and Station 20 in the
Limpopo River basin (see Fig. 3 for the station
locations). Station 24 is the one with the largest drainage area in the
basin with available discharge data. Four stations (highlighted in
Fig. 3) with diverse drainage areas were selected
to assess the influence of the spatial scale and forecast location on the
quality of the forecasts. Table 1 presents the main
characteristics of these stations, such as drainage area, mean annual runoff
and observed runoff coefficient (RC
Model evaluation measures for runoff for selected stations, ordered by basin size.
Upper plots: relative operating characteristic (ROC) diagram representing
false alarm rate versus hit rate for the 6-month SRI (DJFMAM)
Figure 5 (upper plots) presents the ROC diagram for
the 6-month SRI-6
The ROCS of the FS_S4 in predicting SRI-6
ROCS for the SRI-4 (JFMA)
For the high runoff season SRI-4, similar results to those of SRI-6 are observed. In almost every case FS_S4 shows higher skill than FS_ESP and FS_ESPcond. The skill of the forecasts tend to decrease with lead time, especially for FS_ESP and FS_ESPcond, which do not show any skill at the 5-month lead time. In contrast, the skill of FS_S4 for the 5-month lead time is still good. The skill score verification for SRI-1 for the same 4-month period January–April (not shown) shows once more that the FS_S4 is more skilful than the other two forecasting systems. The smaller subbasins (18 and 20) present lower skill for SRI-4 for the three forecasting systems, and while subbasin 18 still presents some skill for all the three FS, subbasin 20 only does so for the FS_S4 for all lead times. In general, for all locations, the skill of the FS_S4 decreases slightly with lead time, while the skill of both FS_ESP and FS_ESPcond decreases more rapidly with lead time. A curious fact is that for a few stations (e.g. 1, 18), for both SRI-4 and SRI-1, the FS_S4 shows a higher skill for a lead time of 4 months than for a lead time of 3 months. For the SRI-4, this means that the forecast is more skilful in predicting the April SRI-4 when issued in December than when issued in January. However, the differences are not statistically significant and this can be due to sampling errors.
Rank histograms for every station and lead time together with the results of the Kolmogorov–Smirnov test show that for the three forecasting systems, uniformity of the distribution cannot be rejected, indicating the forecasts are reliable. Figure 7 presents the rank histograms of SRI-6 for Station 1 for the three forecasting systems as an example.
Rank histograms of SRI-6 for Station 1 for the three forecasting systems (FS_S4, FS_ESP, and FS_ESPcond). The results of the Kolmogorov–Smirnov test for uniformity are presented in each plot.
Figure 8 shows the ROCS and the BSS of the FS_S4 in predicting agricultural drought conditions, i.e. in predicting aggregated RS during the 6-monthly period DJFMAM to be higher than the 70th percentile. Yuan et al. (2013) show that the annual cycle of soil moisture in southern Africa (simulated by the VIC model) lags behind the precipitation. Figure 8 shows that the skill of the FS_S4 forecast in predicting agricultural droughts is higher than climatology (ROCS > 0.5, BSS > 0) throughout the entire basin.
To assess the skill of the seasonal forecast in predicting a specific decision variable in the Limpopo River basin, we calculate the skill of the forecast in predicting water level thresholds in the reservoir that would result in curtailment to the irrigation sector. The availability of water is represented in each cell by the water level. In the cells corresponding to reservoirs, the water level is a surrogate for the storage and is described as a percentage of the full storage capacity of the reservoir. Figure 9 presents the ROCS and the BSS of the FS_S4 in predicting water levels during the 6-monthly period DJFMAM to be lower than the 50th and 37.5th percentiles, based on the analysis described in Sect. 2.4. The figure shows that the skill of the FS_S4 forecast in predicting low water levels is higher than climatology (ROCS > 0.5, BSS > 0) throughout the basin. The spatial distribution across the basin does show the skill to be higher in the northern basin than in the southern basin, which may contribute to the lower skill found at Station 24 close to the basin outlet than at Station 1 in the upper (northern) basin.
The skill scores in cells that contain the reservoirs are represented by a circle to enhance visibility. It is clear from the figure that the skill of the forecast in predicting low water levels is higher in the reservoirs than in nearby streams. This can of course be expected due to the higher memory introduced by the reservoir's storage capacity with respect to the streams. Figure 10 presents the forecast probability of water levels to be lower than the 50th and 37.5th percentiles during the December 1991–May 1992 season as an example. This was the driest season in the last 30 years. The forecast is issued in December 1991.
The forecast probability of water levels in the reservoirs being lower than the 50th and 37.5th percentiles can be interpreted as the forecast probability of a curtailment of 20 and 65 %, respectively, in the irrigation sector during the season. For several reservoirs in the basin the FS_S4 forecast issued in December 1991 predicted a high probability of curtailment to the irrigation sector during the December 1991–May 1992 season. Records confirm the lower than normal water levels during this season, with the irrigation quota indeed being curtailed (DWA, 2013).
Yuan et al. (2013) note that “The major source of seasonal forecast predictability comes from the ocean, and the strongest signal is the El Niño Southern Oscillation (ENSO)”. Given that the ECMWF S4 is influenced by the ENSO signal, it is interesting to analyse how the FS_S4 predicts streamflow in the onset of two clear El Niño years. The 1997/98 El Niño year is described in Thomson et al. (2003) as the largest for this century, predicted with a high degree of certainty. Although many of the climate anomalies typical of an El Niño event took place around the globe, the devastating drought that was feared for southern Africa did not happen (Thomson et al., 2003). For this analysis another year was selected that had a less strong ONI but that did result in a severe drought (1982/83). Figure 11 presents the ensemble seasonal streamflow prediction from FS_S4 for both the 1997/98 and the 1982/83 seasons issued in October and updated in December for Station 24. The plots also show the climatology of the streamflow and the 30th percentile, i.e. the value below which 30 % of the observations are found. The reference streamflow for that season and the forecast ensemble mean are also shown.
Figure 11 shows that in October the predictions from the forecasting system FS_S4 for El Niño seasons of 1982/83 and 1997/98 were relatively similar (see Fig. 11 upper panels) even though the 1997 JAS ONI was notably higher than the 1982 JAS ONI. The updated forecast in December, however, shows a different situation: while the forecast for the 1982/83 season point towards very dry conditions, the forecast of the 1997/98 season indicates near-normal conditions. Yet, the 1997 SON ONI is markedly higher than the 1982 SON ONI. Thus, in spite of the strong ONI conditions, the S4 system correctly forecasted the no-drought condition in the 1997/98 season. This indicates that even though the S4 forecasting system is influenced by the sea surface temperatures over the Niño-3.4 region, the precipitation and temperature forecasts over the Limpopo region are not only constrained by the sea surface temperature evolution but results from the atmospheric circulation response to different climate forcing.
The performance of the three hydrological forecasting systems constructed with the same hydrological model and different meteorological ensemble forecasts are evaluated by means of widely used probabilistic verification skill scores, including the ROC diagram and the rank histogram. Among the forecasting systems considered in this study, FS_ESP is considered the most traditional. Such traditional approaches for hydrological forecasts rely on historical observations of the meteorological conditions, without considering meteorological forecasts. In ensemble probabilistic forecasting, the ESP approach, implicitly accounting for hydrologic persistence and historical variability of climate, is normally used (Brown et al., 2010). FS_S4 is a more complex forecasting system as it requires as forcing the outputs of a seasonal meteorological forecast system, which are complex numerical models and resource intensive. FS_ESPcond, a modification of the ESP approach, conditions its ensemble on past years that had similar climate conditions to the year in which the forecast is made (Brown et al., 2010). Given that the Limpopo region is known to be affected by ENSO and droughts tend to occur during El Niño years, the forecast ensemble was constructed by assigning weights to the different ensemble traces based on the El Niño index.
ROCS and BSS for DJFMAM RS > 70th percentile for the FS_S4.
The skill evaluation of the seasonal forecasts is limited by the use of model data as verification, i.e. we verify our forecasts against the baseline simulation, which was also used to provide the initial conditions to the forecasts. This is the same approach as taken in Yossef et al. (2013), Winsemius et al. (2014), Shukla et al. (2013) and Renner et al. (2009), and while it allows for the detailed (spatial) evaluation of the skill of the forecasts, it can potentially hide limitations of the modelling system. Therefore, these skill results should be interpreted as the upper limit of real predictability of the current system. Results of the seasonal streamflow prediction show that for every lead time FS_S4 is skilful in predicting SRI-6, SRI-4, and SRI-1 during the summer rainy season, while for the other FS the skill is lower and decreases more rapidly with lead time. This means that the complex S4 seasonal forecasting system adds value to the hydrological predictions compared to the climatology-based forecasting systems, as well as the ENSO-mode-conditioned climatology forecast systems. This was also observed during a specific event where expected anomalies due to El Niño did not materialise, but FS_S4 detected this. The skill decreases when going from SRI-6 to SRI-4 and SRI-1. This is as expected given the higher variability of the predictand for shorter aggregation periods. The skill from FS_ESP is lower than that of FS_S4 in almost every case, while the skill of FS_ESPcond is in general between the other two. For SRI-4, FS_ESP and to a lesser extent FS_ESPcond do not show any skill for a 5-month lead time at any of the stations considered.
ROCS and BSS for DJFMAM water level (WL) < 50th percentile (upper plots), and WL < 37.5th percentile (lower plots) for the FS_S4.
Forecast probability of WL < 50th percentile (upper
plots), and WL < 37.5th percentile (lower plots) for the
FS_S4 during the season December 1991–May 1992 issued in Dec 1991
(left panels) and what actually occurred: 1
Seasonal forecast FS_S4 for 2 seasons issued in October (upper panel) and December (lower panel).
As expected, the skill of all forecasts tends to decrease with lead time. This is, however, especially the case for FS_ESP and FS_ESPcond where the decrease in skill with lead time is larger than for FS_S4. For the smaller aggregation periods (SRI-4 and SRI-1) FS_ESP deteriorates to climatology already at a lead time of 3 months for stations 18 and 20, the upstream basins of which are smaller in size. In the larger basins FS_ESP shows predictability up to a 4-month lead time, probably due to the spatial aggregation taking place over larger basins smoothing out uncertainties in space. This indicates that the memory in the hydrology (storage in groundwater, reservoirs, channels and wetlands) contributes to the predictability with a lead time of up to 2–4 months. For longer lead times, the meteorological forcing dominates the predictability of the system. The critical lead time after which the importance of the meteorological forecast exceeds that of the initial conditions depends on the location and size of the basin and should be analysed for each subbasin of interest. Rank histograms for every station and lead time indicate that the three forecast systems are reliable given that uniformity of the distribution cannot be rejected.
The high predictability of FS_S4 for all lead times and aggregation periods of SRI is encouraging given that such a system, if made operational, may provide end users with sufficient time to decide upon measures to take in anticipation. For example, they might decide to change the cropping date or the cropping area if they expect not to have enough water to fulfil the crop requirements. Therefore, there is added value to using a seasonal meteorological forecast (ECMWF S4) to force the hydrological forecasting system when compared to the conventional ESP. The higher skill of the FS_S4 and FS_ESPcond compared to that of the FS_ESP for every lead time is in line with the study of Shukla et al. (2013), who show that for the region of the Limpopo River basin the meteorological forecast dominates the hydrological predictability for the wet season for almost every lead time considered. Only for the 1-month lead time forecasts issued in October did they find a higher influence of the hydrological initial conditions to some extent. Moreover, Yossef et al. (2013) indicate that for semi-arid regions the initial conditions do not contribute much to the skill given the high sensitivity of the runoff coefficient to rainfall variability.
The FS_S4 was also evaluated regarding its ability to predict agricultural droughts and curtailments in irrigation (water levels lower than the 50th and 37.5th percentiles). Maps of spatially distributed ROCS and BSS (Figs. 8, 9) show that the skill of the FS_S4 forecast in predicting these conditions is higher than climatology (ROCS > 0.5, BSS > 0) throughout the basin. Indicating the probability of curtailment to the irrigation sector during the following season is an example of providing a forecast in an understandable format that is useful to the end users. If they are informed that there is a high probability of a high curtailment to the water available for irrigating their crops during the following season, users would have a clear idea of what is the best practice for that situation. Further improvements in forecasting skill could be achieved through better meteorological predictions or better estimation of initial conditions (Yossef et al., 2013). Whether the forecasts indeed have value will depend on the costs of decisions made in response to the forecast, losses in case of a wrong decision and the gain in case of a good decision. This should be further analysed in a continuation of this study.
As a next step, it is recommended that the forecast skill of the FS_S4 and FS_ESPcond be assessed in an actual forecasting mode for a following summer season. The seasonal meteorological forecast from S4 can be obtained in real time for research purposes. To test a pre-operational system, the forecasting system ought to be statistically post-processed in order to remove biases in streamflow predictions. Moreover, the initial conditions for the forecasts could be better estimated through data assimilation of water levels in reservoirs and streams. This data could be obtained from the water managers of the basin. Despite the limitations of FS_S4 (access to real-time atmospheric–ocean seasonal forecasts for non-ECMWF member-states, and their quality) and FS_ESPcond (depending on the calibration and decreased skill at long lead times), both systems show potential for seasonal hydrological drought forecasting in the Limpopo River basin to provide operational guidance to users.
We evaluate the performance of three forecasting systems (FS_S4, FS_ESP, and FS_ESPcond) in the Limpopo
River basin. These systems make use of the same hydrological model and are
forced with three different meteorological ensemble forecasts (two of which
are based on resampled climatological records, FS_ESP and
FS_ESPcond, and one based on seasonal meteorological
forecasts, FS_S4). Results of the seasonal streamflow
prediction show that the three forecasting systems show moderate skill in
predicting SRI-6 (DJFMAM)
For every lead time and aggregation period considered, FS_S4
is found to be skilful in predicting hydrological droughts represented by
SRI
The high predictability of drought of FS_S4 for all lead times and aggregation periods of SRI and for the spatial drought indicators is encouraging given that such a system, if made operational, may provide end users with sufficient time to decide upon measures to take in anticipation. Moreover, FS_ESPcond shows promising results. This forecasting system only requires the ONI index previous to the forecast to weight the ensemble traces to include in the forecast. This system is relatively simple and presents the advantage that it can be coupled with the forecast of the ONI index that is available with a long lead time. Naturally, in this situation the uncertainties of both forecasts need to be considered.
This study was carried out in the scope of the DEWFORA (Improved Drought Early Warning and Forecasting to strengthen preparedness and adaptation to droughts in Africa) project which is funded by the Seventh Framework Programme for Research and Technological Development (FP7) of the European Union (grant agreement no. 265454). Edited by: H. Cloke