Detection of environmental change is a huge area of research which cannot easily be reflected in an introduction. More extensive reviews of change detection methods in hydrology are available (e.g. Yue et al., 2012) and there are textbooks on trend testing in the environmental sciences in general (e.g. Chandler and Scott, 2011). The overview below will give the reader a flavour of the methods which are available, with a brief critique, to set the new method described in Sect. 1.2 in context. The choice of change detection method clearly depends on the users' aims and available data.

The majority of hydrological change detection studies use monotonic trend tests such as Mann–Kendall (details of which can be found in Yue et al., 2012) which are influenced by the amount of autocorrelation in the data as well as by the start and end points of periods to which the trends tests are applied (Hannaford et al., 2013; Chen and Grasby, 2009). This is particularly problematic when the gauging stations have relatively short records starting in a dry or wet period. For example, the UK gauging station network was largely built in the 1960s when the North Atlantic Oscillation Index (NAOI) was in a strong negative phase resulting in conditions for the UK which were drier than much of the following record. Furthermore, monotonic trend tests only provide information as to whether change has occurred over the time period being investigated and no information is gained as to the type (e.g. abrupt or gradual) or the timing of change. This is a major limitation as it makes it difficult to link a simple monotonic trend in streamflow to trends in potential drivers of change (i.e. changes in meteorological conditions or catchment properties). A further weakness of current change detection methods is that they often use indicators of flow selected a priori to characterise a particular aspect of the flow regime (e.g. the Q95; 7 day minimum flow; frequency of peaks-over-threshold), which potentially introduces bias by selecting a pre-determined aspect of the flow regime.

Another approach to change detection is change-point analysis, which can be used to identify the temporal location where change occurs (e.g. Beaulieu et al., 2012, applied change-point analysis to climate variables and Jandhyala et al., 2013, reviews change-point analysis including a plethora of studies which investigated change points in the Nile river flow time series). Change-point analysis identifies the temporal location at which one or more properties of the river flow time series change abruptly (e.g. a change in the magnitude, variability or autocorrelation, etc.) but is associated with several limitations. Firstly, there is increased uncertainty about change points detected close to the start or end of the time series (due to a higher risk of false detection). Secondly, the method only detects changes in one aspect of the time series (e.g. linear trend, magnitude, variability or autocorrelation). Finally, although change-point analysis is designed to detect abrupt changes there is, in practice, great difficulty in discriminating between trends and abrupt changes (as demonstrated by Rougé et al., 2013). Jarušková (1997) provides a cautionary review of change-point detection methods for river flow data.

An alternative approach to change detection is through analysis of periodicities. There is a wide range of methods available for decomposition of time series into various components (e.g. Fourier methods, empirical mode decomposition, wavelets; see for example Labat, 2005 and Sang, 2013). These approaches can detect complex non-linear patterns of variability and do not require the selection of indicators as they are normally based on the whole time series. However, such approaches normally characterise periodicities over a range of scales, rather than changes over time. It is hard to relate the change in spectral shape to the hydrological regime (Smith et al., 1998). This is indicated by recent studies in the UK which applied these methods and did not go beyond looking at the high-level drivers, particularly the NAOI (e.g. Sen, 2009; Holman et al., 2011). Similarly, Kumar and Duffy (2009) use single spectral analysis to look at the precipitation–temperature–river flow relationship. This analysis enabled the authors to link the identified temporal changes to the southern oscillation as well as large anthropogenic influences (dam building and pumping), but did not investigate how changes in different aspects of the precipitation regime (e.g. seasonality and magnitude) influence the river flow time series.

Here a novel and fundamentally different methodology for detection of hydrological change is introduced using variograms that are applied to moving windows in a river flow time series (hereafter, temporally shifting variograms, TSVs). The TSV method provides insights into how river flow dynamics evolve through time, without relying on fixed study periods or pre-determined flow indicators. This enables streamflow changes to be linked explicitly with external drivers (e.g. meteorological forcing). Variograms are able to capture the temporal dependence structure of the river flow (i.e. on average, how dependent river flow on a particular day is on river flow on the preceding days). The temporal dependence structure is closely related to the amount of variability at different temporal scales in the time series and, as it is influenced by catchment characteristics (Chiverton et al., 2015), it enables inferences to be made about the precipitation-to-flow relationship in a catchment.

As previously noted in the introduction there are several methods of identifying temporal changes in river flow and a large range of indicators which could also be investigated using a moving window. The TSV has key advantages over existing methods. Firstly, the variogram can be thought of as a composite indicator which provides information about a range of aspects in the river flow time series, hence enabling a range of possible temporal changes in river flow dynamics (e.g. standard deviation and seasonality) to be captured. Variograms can also detect changes in daily river flow which other indicators may not be able to (e.g. changes in variability at a range of time scales). Furthermore the variogram is calculated using daily flow data and does not rely on the user extracting pre-conceived aspects of the river flow regime via the calculation of indicators (e.g. annual or seasonal averages, minimum or maximum flow). This enables the whole flow regime to be investigated, rather than much of the daily flow information being discarded, as is the case when calculating some indicators (e.g. annual 7 day minimum flow).

It is worth noting that there are a range of stochastic techniques which can characterise the basic autocorrelation structure of data (e.g. AR, ARIMA, etc.). These classical time series analysis approaches have been widely used to investigate hydrological behaviour (e.g. Salas et al., 1982; Montanari et al., 1997; Chun et al., 2013). Such approaches characterise temporal dependence and can also in principle be applied in moving windows (e.g. AR1 applied in 20-year moving windows by Pagano and Garen, 2005). A limitation with the classical models is that the user has to select the appropriate AR and MA parameters, a potentially subjective process, which will vary between catchments. In practice, they have not been widely used to examine changes in temporal dependence through time.

The method we propose uses variograms to characterise the autocorrelation so that the AR parameter does not need to be specified. Furthermore, variograms are designed to handle missing data which is common in river flow time series. The variogram has several defined parameters (e.g. nugget, sill and range) which characterise different aspects of the autocorrelation structure that can be used in moving window change analysis. This enables changes in several aspects of the river flow regime to be analysed in parallel.

Conventionally most trend analysis studies focus on change detection, and attribution is often based on qualitative reasoning and relies on published work to support the hypothesis (Merz et al., 2012). The TSV method enables changes in river flow (associated with changes in variogram parameters) to be quantitatively related to meteorological characteristics. This work is an attempt to provide a formal “proof of consistency” (Merz et al., 2012) that river flow changes can be associated with changes in meteorological drivers. This is an important new development, as few published studies of streamflow change have sought to explain observed patterns through links to precipitation. We acknowledge that this does not amount to full attribution without “proof of inconsistency” with other drivers (e.g. land use change), but it does provide a solid foundation for such attribution studies. In principle, the method could be used with a wider range of drivers, both natural and anthropogenic, if temporal data on, e.g. land-use change, were also available.

This study has the following objectives: develop a novel change detection method (TSV) to detect both linear and non-linear changes throughout the river flow regime; test the performance of the method by imposing artificial changes to a river flow time series; identify patterns of temporal change in rivers for a set of 94 catchments in the UK; and explain the contribution of precipitation to the detected variability in variogram parameters. This paper is structured as follows: Sect. 2 describes the data employed; Sect. 3 details the TSV method; Sect. 4 tests the TSV method using artificially perturbed river flow time series; Sect. 5 identifies the periods of change across the 94 UK catchments and Sect. 6 investigates the meteorological drivers.

Near-natural UK benchmark network catchments, with only modest net impacts from artificial influences, were chosen (Bradford and Marsh, 2003). These catchments are deemed to have good data quality and therefore artificial influences will be limited. Furthermore, only catchments with a record length of 33 years or more (1980–2012) of daily river flow data with less than 5 % missing data were considered. Nested catchments with similar flow regimes were excluded.

This data set was used in a previous study which classified UK catchments into four classes according to their average temporal dependence structure (Chiverton et al., 2015). One of these classes was excluded from the present study; this comprises catchments which have high infiltration and storage, hence with distinctly different precipitation-to-flow relationships than the rest of the catchments. In particular, Chiverton et al. (2015) demonstrated that these catchments have very long-lasting temporal autocorrelation of over a year, largely due to the influence of groundwater storage, instead of weeks to a few months like the other catchments. To avoid this very different catchment response time overly influencing results, catchments which overlay highly productive aquifers were removed (mainly in the south-east of England). This resulted in 94 catchments, shown in Fig. 1.

Daily catchment-averaged precipitation values were calculated from CEH-GEAR, a 1 km2 gridded precipitation data set (Tanguy et al., 2014) derived using the method outlined in Keller et al. (2015). From this data, characteristics which represent different aspects of the precipitation regime were calculated (Table 1).

An overview of how the river flow time series has been de-seasonalised and standardised (steps 1 to 5) is provided here, but in-depth discussion can be found in Chiverton et al. (2015).

The river flow data were in-filled, using the equipercentile linking method (Hughes and Smakhtin, 1996), to remove periods of missing data. This was required to improve the de-seasonalisation (step 3).

The temporal dependence structure can be represented by a one-dimensional temporally averaged variogram (see Chandler and Scott, 2011, or Webster and Oliver, 2007, for a detailed background on variograms). Based on the transformed, de-seasonalised standardised flow data, an empirical semi-variogram was calculated for each catchment using the average squared difference between all pairs of values which are separated by the corresponding time lag (Eq. 1 which calculated the semi-variance): v^h=12(N-h)∑i=1N-h[(Yti+h-Yti)2], where h is the lag time, Y(ti) is the value of the transformed data at time ti and (N-h) is the number of pairs with time lag h.

A variogram model was then fitted using the variofit function (from the geoR package in R and the Cressie method; Cressie, 1985) to the empirical semi-variogram to enable the following parameters to be calculated (Fig. 2): the nugget, which is the y intercept, represents a combination of measurement error and sub-daily variability; the sill is defined as the semi-variance where the gradient of the variogram is zero. A zero gradient indicates the limit of temporal dependence and is an indicator of the total amount of temporally auto-correlated variance in the time series. The partial sill is the sill minus the nugget and shows the temporally dependent component, used herein as the sill. The range is the lag time at which the variogram reaches the sill value. Autocorrelation (gradient of the variogram) is essentially zero beyond the range. The practical range is the smallest distance beyond which covariance is no more than 5 % of the maximal covariance (time it takes to reach 95 % of the sill) (Journel and Huijbregts, 1978). As the variogram is only asymptotic to the horizontal line which represents the sill, the practical range is used herein as the range.

The fundamental premise of the TSV approach is that variograms are applied in moving windows through a time series, to determine the extent to which variogram properties (which characterise the autocorrelation structure) change through time. To examine how unusual these changes are in the context of the observed streamflow record, the method determines whether variogram properties in each window are outside thresholds which encompass the 5–95 % of expected values based on the original 32-year average variogram. Periods of change (compared to the 32-year average variogram) were thus detected for the 94 catchments using the following method, applied to each catchment:

Compute bootstrap parameter estimates from multiple realisations of the 32-year average variogram, which are created by simulating 1000 standardised river flow time series assuming a Gaussian random field model (see Havard and Held, 2005 for more detail). The data were simulated using the model parameters from the original 32-year variogram, so the output has the same lags as the original data (i.e. daily). A variogram was then created for each of the time series.

Four variogram parameters were calculated. Firstly, the sill and range were calculated. However, as the data used are relatively high frequency (daily) and good quality, the value for the nugget is low (although not zero as there is measurement error and sub-daily variability) and the 5th percentile is zero. Therefore, the nugget cannot be handled in the same way as the other variogram parameters (i.e. decreases below the lower bound cannot be investigated). Instead, a new parameter, the 3-day average semi-variance (3 DASV) (average of the first three points of the semi-variogram) was defined and used to investigate changes in very short term temporal dependence. A further parameter was defined, the half-range average semi-variance (HRASV) (average of the points up to half the practical range to provide information on the intermediate temporal variability (between the 3 DASV and the sill, which is the total amount of auto-correlated variability).

It is acknowledged that there is uncertainty surrounding the variogram calculated from the river flow data. Part of the uncertainty comes from river flow measurement and part from the fitting of the variogram model. Due to the number of catchments and moving windows it is beyond the scope of this paper to do a full uncertainty analysis as discussed in Marchant and Lark (2004). Therefore a stability test was carried out in order to verify if the changes detected in the TSV method are caused by a change in the autocorrelation structure or by a few extreme points influencing how the variogram model fits the data. This is usually undertaken by doing a split test. However, due to the requirement of having a large data set to calculate the variogram, splitting the 5-year moving window in two was not deemed appropriate. Instead each data point in the 5-year moving window was randomly assigned to one of ten equal sized groups. The variogram was then fitted to the data 10 times, each time removing the data from one of the groups meaning that the variogram was fitted to 90 % of the data. This resulted in 10 values for each variogram parameter which were calculated using 90 % of the data. These points were then plotted against the variogram parameters which were calculated using 100 % of the data to provide an indication as to the stability of the variogram parameter estimates.

Having established patterns of temporal variability using the TSV approach, the potential meteorological drivers behind the detected changes in the variogram parameters were identified before being used to calculate how much of the change they explain.

Firstly, Pearson's product-moment correlation was calculated between the time series of each of the four variogram parameters and the time series of precipitation characteristics, calculated over the same time window. These results were used to determine the likely drivers behind each variogram parameter.

Secondly, multiple linear regression (MLR) was undertaken in order to determine how much variance in the variogram parameters could be explained by a combination of different precipitation characteristics. As precipitation characteristics are correlated with each other, a procedure which penalises extra model parameters was required. Stepwise regression, which tests whether parameters are significantly different from zero, has limitations – in particular, it can lead to bias in the parameters, over-fitting and incorrect significance tests (see Whittingham et al., 2005, for an in-depth discussion). In addition, the number and order of the potential parameters can influence the final model (Burnham and Anderson, 2002). Instead, information theory (IT) based on Akaike's information criterion (AIC) was used to analyse how much information is added by each characteristic. For each catchment the model with the lowest AIC score was used to obtain the R2 value which provides an indication into the amount of change in the variogram parameters which can be explained by precipitation.

The relative importance of each precipitation characteristic was also investigated, providing information on which precipitation characteristics are important in explaining the changes in each variogram parameter. The relative importance was obtained by calculating the R2 contribution averaged over orderings among regressors for each precipitation characteristic using the Lindeman–Merenda–Gold (LMG) method proposed by Linderman et al. (1980), as recommended by Gromping (2006).

Positive autocorrelation would influence the efficiency of the explanatory variables causing an overestimation of the significance. However, analysing the residuals from the MLR between precipitation and river flow (using the Durbin–Watson test for autocorrelation disturbance) showed no significant autocorrelation. Therefore, regressing against several precipitation variables with similar autocorrelation to the variogram parameters (both averaged over 5-year moving windows) was deemed to adequately remove the autocorrelation.

Before identifying the temporal changes, the stability of the variogram parameters was analysed to investigate if certain data points have a large influence on the shape of the variogram and hence the variogram parameters. Figure 5 shows the relationship between the variogram parameters which are calculated using 100 % of the available river flow data and the same parameters calculated using 90 % of the available data. The figure highlights that there is a strong relationship between the points calculated using 90 and 100 % of the data. However, there are points which deviate greatly from the x=y gradient. The red dashed lines in Fig. 5 represent small deviations from the x=y plot which are deemed to be an acceptable amount of variation due to the removal of 10 % of the data. Any catchment which has a point or more outside these lines, for any variogram parameter, was removed. This resulted in three catchments being removed from subsequent analysis (reducing the number of catchments from 94 to 91). As well as the points outside of the red dashed lines, the range has two groups of values that exceed the length of the red dashed lines (catchments with a range of over 170 days). These two groups have large variability in the 10 values containing 90 % of the data. The large variability is probably due to the extrapolation by the model from the calculated semi-variance. Due to the fact that all the values are above the 95th threshold (and therefore it is likely that they capture a true change in the range) these values were retained.

Figure 6 identifies the periods when the TSV characteristics go above or below the 95th or 5th percentiles from the average variogram, respectively, for the 91 catchments. Different variogram parameters exhibit different changes through time. The 3 DASV shows relatively little change, until after 2004 when there is a peak in the number of catchments above the upper threshold. The sill has peaks in the number of catchments going above the upper threshold around 1980, 1990 and after 2004. The range and the HRASV show several periods where the number of catchments above the upper threshold is much greater than the number of catchments below the lower threshold and vice versa. The range and the HRASV see dramatic increases in the number of catchments which go beyond the lower and upper thresholds respectively, during approximately 1995–2001. Throughout this period the total amount of variability (the sill) remains the same, as does the 3 DASV. The medium-term variability (HRASV) shows an increase. The length of time the temporal dependence lasts (the range) decreases. In addition to the 1995–2001 period, every variogram parameter exhibits an increase in catchments exceeding the thresholds after around 2004. This indicates increases in the total (sill) and short- to medium-term (3 DASV and HRASV) variability in the river flow time series.

Initial analysis investigated the difference in precipitation between the periods which show the greatest changes, in terms of the number of catchments which go below/above the thresholds (approximately 1995–2001 and 2004–2012), with the preceding time series (1980–1994). The periods where the most exceedances occur (1995–2001 and 2004–2012) are significantly more variable than the preceding time series (Table 2).

To explore the links with drivers more quantitatively, the relationship between precipitation characteristics and variogram parameters in the 5-year moving windows was calculated, with the results summarised for all catchments in Table 3.

The sill has the largest relationship with the winter to summer ratio (negative) followed by the standard deviation (positive). Although these appear contradictory, closer inspection found that the winter value seldom changed whereas the summer value increased (decreasing the winter to summer ratio), increasing the sill. The range is most correlated with the lower percentiles (negative) and the length of wet and dry periods (negative and positive respectively). Similar to the sill, the 3 DASV has the largest correlations with the standard deviation (positive), winter to summer ratio (negative), mean (positive) and 90th percentile (positive). The largest correlations are with the HRASV which is highly correlated with the percentiles (positive), SD (positive) and the mean (positive).

Each variogram characteristic has a different relationship with the precipitation characteristics (Table 3). As expected from the artificial analysis (Fig. 4) the sill, HRASV and 3 DASV are more influenced by precipitation characteristics which affect the short-term or total amount of variability in the time series (e.g. standard deviation and the different percentiles). The range is most influenced by aspects of the precipitation which enhance correlation between the river flow on successive days (e.g. length of wet and dry periods). The relationship between the precipitation characteristics and the range is usually in the opposite direction to the other variogram parameters.

The average relative importance of each indicator in predicting each variogram parameter was calculated using the LMG method. The three most important characteristics for the sill (accounting for over 30 % of the explained variance between them) are the winter to summer ratio, standard deviation and 90th percentile. The three most influential characteristics for the 3 DASV were the same as for the sill. The average length of time below and above 1 mm accounts for over 30 % of the explained variance for the range. For the HRASV, standard deviation, winter to summer ratio and the mean precipitation account for over 30 % of the explained variance. Although these key drivers have been identified, the total amount of variability in the variogram parameters which is explained by precipitation characteristics is mixed and depends on both the variogram parameter and the catchment, as shown by the spread of values of explained variance for individual catchments (Fig. 7).