Hydrological models have numerous applications of central importance to society including for planning, design, and management of environmental and water resources. The operation of hydropower systems is mainly constrained by the availability of water resources. Hydrological models play an important role in forecasting the local inflows to the system on scales ranging from hours to years. With due recognition of the need for accurate prediction of streamflow and snow storage, Statkraft (2018) has recently released a new modeling framework mainly tailored for an operational purpose. In this study, one of the conceptual models of this framework was subjected to uncertainty analysis. Conceptual hydrological models typically have one or more calibration parameters and commonly require some form of inverse modeling to estimate model parameters from observations (Crawford and Linsley, 1966). During calibration, equifinality arises when different parameter sets give equally good results in terms of predefined efficiency criteria (Beven, 1993; Savenije, 2001; Wagener et al., 2003). The generalized likelihood uncertainty estimation (GLUE) methodology (Beven and Binley, 1992) is an extension of the generalized sensitivity analysis concept of Hornberger and Spear (1981), and it accepts equifinality as a working paradigm for parameter calibration of hydrological models (Choi and Beven, 2007). It is based on the concept that all models of hydrological systems are highly simplified representations of reality (e.g., Reichert and Omlin, 1997), and hence it is expected to have several different model structures and parameter sets that describe the system in an adequate way (Blazkova and Beven, 2002). When dealing with nonlinear systems, the classic hydrological approach of using a single set of model parameters may lead to large predictive biases (e.g., Mantovan and Todini, 2006).

Hydrological modeling is affected by four main sources of uncertainty related to input data, validation data, model structure, and model parameters (e.g., Renard et al., 2010). Input data uncertainties may arise from measurement limitations and scaling issues, for example, due to forcing data downscaling. Errors of the rating curve affect streamflow estimates and thereby lead to validation data uncertainty. Structural uncertainty may result from the underlying assumptions and simplifications in the model formulation as well as from application of the model to conditions inconsistent with the model structure (Tripp and Niemann, 2008). Parametric uncertainty reflects the inability to specify exact values of model parameters (Renard et al., 2010) and it may stem from errors in input data and observations used for model conditioning as well as be due to epistemic errors in model structure. An increased awareness of these modeling uncertainties and the need for quality control of such models requires the integration of uncertainty analysis into the modeling process from the very beginning (Beven, 1989; Saltelli et al., 2006; Refsgaard et al., 2007).

Uncertainty analysis techniques can be classified as frequentist or Bayesian approaches, probabilistic or non-probabilistic approaches (e.g., Montanari et al., 2009), or as formal or informal approaches (e.g., Vrugt et al., 2009). Among the most widely used techniques in hydrological modeling are the formal Bayesian and the GLUE methods (Jin et al., 2010). The formal Bayesian approach makes strong assumptions about the statistics of observed data; with the likelihood function defined based on assumptions about the nature of the residuals (Schoups and Vrugt, 2010). However, the choice of an adequate likelihood function has been the subject of considerable debate. According to Beven and Smith (2015), a formal probabilistic likelihood function will have limited value since non-stationary epistemic uncertainties cannot be adequately represented by a statistical model. In GLUE, the likelihood measure is associated with a parameter set and should ideally reflect all the different sources of uncertainty (Beven and Smith, 2015). The original GLUE methodology has been the subject of debate for using a subjectively set threshold of behavioral models (e.g., Mantovan and Todini, 2006; Stedinger et al., 2008; Clark et al., 2011; Nearing et al., 2016). This problem is common to most residual-based model selection methods (Schaefli, 2016). The extended concept of behavioral models in the GLUE limits of acceptability approach (GLUE LOA) (Beven, 2006) attempts to overcome this drawback through use of error bounds of the observational dataset.

The GLUE LOA methodology involves specifying limits around some observational data within which model predictions are required to lie and thereby considered acceptable for the intended model application. The acceptability limits are set prior to running a model and, among other considerations, they are expected to take into account incommensurability and uncertainty in both the input and evaluation data (Beven, 2009). However, identification of models that reproduce the observed system behavior within the limits of measurement error is not easy due to time-varying errors in the input data and model structure (e.g., Beven, 2016). This difficulty is even more pronounced when input and other sources of errors are not explicitly accounted for in defining the LOA.

Good quality time series data and associated uncertainties are not always readily available. For example, in regulated catchments the inflow hydrograph is often estimated from changes in storage volume and outflows using the water balance equation. Thus, as in the case of our study catchment, no stage–discharge relationship exists for estimating the streamflow uncertainty using the usual practice, i.e., by fitting different rating curves. In such instances the alternative is to assume an observation error proportional to the observational data. However, the identification of behavioral models without due consideration to such less precise observation error estimates may lead to the rejection of a useful model (i.e., making a type II error). Some of the measures taken to minimize the risk of making a type II error when identifying behavioral models using the GLUE LOA include extending the limits (e.g., Blazkova and Beven, 2009; Liu et al., 2009) and using different model realizations for different periods of a hydrological year (e.g., Choi and Beven, 2007). In this study, instead of relaxing the limits, the percentage of observations where model predictions are required to fall within the acceptability limits was relaxed.

The GLUE methodology has been widely used in various disciplines (Beven, 2009; Efstratiadis and Koutsoyiannis, 2010) primarily due to its conceptual simplicity and ease of implementation. Further, its suitability for parallel implementation on distributed computer systems as well as its general strategy in dealing with equifinality in model calibration make it an appealing framework (Blasone et al., 2008; Shen et al., 2012; Mirzaei et al., 2015).

In this study model parameters were constrained using streamflow and the MODIS snow cover product (Hall et al., 2006). Multi-criteria model conditioning helps to reduce prediction uncertainty through improved parameter identification (e.g., Efstratiadis and Koutsoyiannis, 2010; Finger et al., 2015), and GLUE provides a flexible approach for using multi-criteria methods through different ways of combining measures. Besides streamflow, one of the observations commonly used in multi-criteria conditioning of rainfall-runoff models in snow-dominated catchments is snow data. Remote sensing snow cover data have been used in several hydrological modeling studies for deriving and updating a snow depletion curve (SDC) (e.g., Lee et al., 2005; Kolberg and Gottschalk, 2006; Bavera et al., 2012), as well as in multi-criteria-based model calibration and simulated snow cover validation (e.g., Udnaes et al., 2007; Parajka and Bloschl, 2008; Berezowski et al., 2015). However, studies involving combined uncertainty of streamflow and snow cover predictions using the GLUE methodology are still missing in the literature.

The main objective of this study is to assess parameter uncertainty for a recently developed distributed conceptual hydrological model using the GLUE methodology with due consideration to the model's main application as an operational hydrological model. The second objective is to investigate the potential value of snow cover data as additional observation in conditioning model parameters in the study area. The third objective is to assess the possibility of using a time-relaxed GLUE LOA approach for constraining model parameters. In doing so, we employ a novel empirical approach for implicitly accounting for the effects of input and observational data errors by relaxing the percentage of time steps in which predictions of model realizations fall within the limits.

This paper is organized as follows. First the (i) hydrological model and (ii) the study site and relevant data used in this study are briefly described in Sect. 2.1 and 2.2. The procedures followed to set up the uncertainty analyses are then outlined in Sect. 2.3. In Sect. 3, the results from parameter uncertainty as well as the uncertainty of streamflow and snow cover predictions using the residual-based GLUE approach are presented. The results from the relaxed GLUE LOA are also presented in this section. Finally, in Sects. 4 and 5, the analysis results and their implication on the hydrologic model, the data and the methodologies followed are discussed and conclusions are drawn.

The streamflow prediction uncertainty analyses results show that model performance was relatively lower during low-streamflow than high-streamflow conditions throughout most validation periods (e.g., Table 4). A similar result was reported by Choi and Beven (2007) in their multiperiod cluster-based uncertainty analysis in the Bukmoon catchment, South Korea, where a high percentage of simulation bias was observed during the drier seasons due to relatively poor model performance during these periods. The result of this study is thus consistent with the general observation that catchment hydrologic models perform relatively well in wet conditions but break down during low-streamflow conditions (e.g., Kirchner, 2009). In the case of results from the residual-based GLUE methodology, this can also be partly attributed to the nature of the likelihood measure used to identify the behavioral models. The result reveals this observation, where model performance during low-streamflow periods (LnNSE) was improved when using the combined likelihood measures as compared to using NSE alone. This is because models identified using NSE alone strongly reflect the hydrologic characteristics of the high-streamflow periods and are expected to perform more poorly during low-streamflow conditions.

In order to assess the potential value of MODIS SCF in constraining model parameters, the snow sub-model parameters were constrained using this observation and the posterior distribution of the individual parameters were compared against corresponding distributions that resulted from model conditioning using streamflow only. Parameter inference based on SCF only as a conditioning observation gave some parameter estimates that deviate significantly from those obtained when conditioned with streamflow only (Fig. 10). The box plots depict the posterior distribution of the snow-related parameters separately conditioned using streamflow and SCF. For the ease of comparison, parameter values were scaled between 0 and 1. From these plots it can be seen that tx and wind scale are the model parameters most sensitive to the conditioning data type with a significant shift in their quartiles towards the upper part of the parameter dimensions when conditioned using SCF, whereas the fast ADR, slow ADR, and snow CV did not show significant displacement in their posterior distribution. These parameters were also identified as the least sensitive model parameters when the model was constrained using streamflow only.

Generally, in snow models with the sub-grid snow distribution component parameterized using the statistical probability distribution function, low snow CV results in a faster depletion rate of the snow-covered fraction (e.g., Liston, 2004). Thus, the slight displacement of snow CV posterior values towards the lower part of its parameter dimensions coupled with the increased posterior values of wind scale would give rise to lower snow cover fraction during the melting period when model parameters are constrained using SCF only. On the other hand, the increased posterior values of the rain–snow threshold (tx) would result in an increase in snow deposition and thereby in a partial or full canceling out of the effects of changes in snow CV and wind scale. This phenomenon may thus lead to equifinality, where different sets of model parameters give comparable SCF responses.

In the GLUE LOA approach a particular model realization is classified as acceptable if its prediction falls within the limits for all observed values. In continuous rainfall-runoff modeling it is difficult for all predictions of a given model realization to lie within the observation limits in a time series. In some cases this phenomenon can be attributed to different specific processes dominating the hydrologic behavior of a catchment at different sub-periods, while in other instances it may be due to a lack of a viable means for explicitly accounting for the effect of variable sources and level of uncertainties from the input data errors, which are difficult to set a priori. Thus, the time-varying likely effects of other sources of errors such as input errors on prediction uncertainty need also be implicitly taken into account when defining the limits of acceptability.

The use of GLUE LOA for testing hydrologic models as hypotheses without a due consideration of errors in input data may lead to a rejection of useful models that might adequately represent the catchment behavior and thereby to making a type II error (false negative). In the past, various attempts have been made to minimize the risk of making type II errors in model calibration studies using the GLUE and other frameworks. In some studies an improved calibration of hydrologic models was obtained through independent calibration of sub-periods of a time series (e.g., Boyle et al., 2000; Samanta and Mackay, 2003). When it comes to the GLUE LOA approach, extending the limits (e.g., Blazkova and Beven, 2009; Liu et al., 2009) and using different model realizations for different periods of a hydrological year (e.g., Choi and Beven, 2007) are some of the measures taken to minimize the risk of making type II errors. Common to all these measures is that they attempt to relax the selection criteria for behavioral models.

In this study when using the GLUE LOA approach, the streamflow bounds were set to ±25 % and the result shows that none of the model realizations were able to satisfy the LOA criteria without one or more of their predictions falling outside the acceptable streamflow bounds. The failure rate was higher during low-flow conditions as compared to high-flow conditions. An initial attempt was made to relax the limit of acceptability by extending the streamflow bounds. Regardless, no model realization with its predictions falling within the error bounds for all observations was found until the limits were extended to over ±85 %. This relaxed acceptability limit seems less reasonable in terms of its physical meaning as an error bound. Therefore, rather than relaxing the limits, an alternative empirical approach was followed by relaxing the number of simulation time steps which fulfilled the original LOA criterion. The procedure involves defining the acceptable percentage of observations that are required to be bracketed by model predictions (during the calibration period) based on a predefined acceptable CR value.

This empirical approach is based on the observed relationship between prediction uncertainty and number of behavioral models, which in turn is a function of the selection criterion. As the threshold value of a likelihood measure increases (in the case of residual-based GLUE) or absolute value of the limits decreases (in the case of GLUE LOA), the simulated runoff series gradually converges, though not necessarily to the observations. A similar observation was also reported in other GLUE-based uncertainty studies (e.g., Xiong et al., 2008). A further analysis in this study reveals that, as the percentage of observations required to be bracketed by each model realization (pLOA) increases, the number of behavioral models decreases and thereby the simulated runoff series converges, resulting in a low CR (Fig. 11). In this study, the threshold pLOA for each calibration period was defined in such a way that the 5 %–95 % prediction uncertainties of streamflow using the residual and the LOA-based GLUE methodologies are similar. Defining the threshold pLOA this way helps to set a reasonable value that minimizes the risk of making type II errors while maintaining the overall model accuracy by rejecting the inclusion of non-behavioral models. Furthermore, it helps to roughly compare the performance of behavioral models selected using the relaxed GLUE LOA against the residual-based GLUE in terms of their ability to reproduce the median streamflow and SCF predictions at a similar level of uncertainty (i.e., the CR used to set pLOA).

Although it is difficult to single out the effects of input data error from model structural error on model performance using the GLUE methodology, the error patterns may aid in assessing model performance in different periods of the hydrologic year. Generally, a good model structure coupled with good data is not expected to give a consistent bias (e.g., Liu et al., 2009). Figure 12 shows a sample daily percentage of acceptable simulations satisfying the LOA criteria during the hydrologic year 2012. The percentage of the acceptable number of model realizations in each time step was generally low during the calibration period (<65 %). However, for each time step, predictions from some behavioral models are able to mimic the corresponding observation within the assumed error bound. The percentage of acceptable models was relatively higher during high than low-streamflow conditions. And this result is consistent with the general observation that most hydrological models perform relatively well during high-streamflow compared to low-streamflow periods. The spike in the percentage of acceptable models in the month of February 2012 when time steps around are so low, however, reveals how model performances can unexpectedly vary between time steps in response to input data errors and/or the observational error bounds. The observed spike could thus be attributed to relatively low input data errors and/or lower actual observational error bounds as compared to the assumed average values for the particular time step. The distribution of the behavioral model weights over the calibration period shows that the mean weight during the period where the spike occurred is very low. Similarly the median weight of behavioral models during this period is close to zero, implying that most of the model realizations have their predictions that barely fall within the limits.

This result reveals that the GLUE LOA with relaxation in percentage of observations where model predictions fall within observational error bounds can be used as an alternative approach for conditioning model parameters and conducting an uncertainty analysis when there is a lack of metadata on input and observational data uncertainty coupled with a highly time-varying level of uncertainty from such sources. After relaxation, a limited sample of the total observations, i.e., 30 %–40 % of a hydrologic year, was able to effectively identify behavioral models, and this result is consistent with findings of other studies dealing with the effect of observation size on constraining model parameters (e.g., Seibert and Beven, 2009; Liu and Han, 2010; Sun et al., 2017). The relative accuracy of an event and other factors that affect the information content of the input and observation datasets (e.g., Beven and Smith, 2015) are more important than the length of the datasets, especially in continuous rainfall-runoff modeling.

Two GLUE methodology variants were applied for parameter uncertainty analysis of a distributed conceptual hydrological model. The analysis result from the residual-based GLUE methodology shows that the catchment response parameters, viz. c1, c2, and c3 as well as the wind scale, are the most sensitive model parameters. More caution is thus required when defining the value range of these parameters. On the other hand, the fast and slow albedo decay rates as well as the snow CV are relatively more uncertain model parameters.

Model conditioning using combined streamflow and MODIS SCF did not improve the median prediction of streamflow as compared to the result when model parameters are conditioned using streamflow only. A similar result was also observed for SCF predictions. The additional information from the MODIS SCF data was generally less significant in constraining the rainfall-runoff model parameters.

When using the GLUE LOA approach, the model did not provide any behavioral simulation that yields predictions within the assumed observational error bound in over 90% of the time steps. A relaxation was needed in order to partly overcome the limitations of using constant observational error proportionality and not taking an explicit account of the other sources of uncertainty such as from input data errors. A relaxed GLUE LOA approach was introduced that allows a relaxation on the number of time steps required to achieve the LOA. Similar results are obtained using both the residual-based GLUE and the relaxed GLUE LOA approaches. Relaxing the percentage of observations required to be bracketed per simulation period by a particular model realization (pLOA) was found to be more effective than relaxing the observational error bounds. In this study the 5 %–95 % prediction uncertainty of the residual-based GLUE methodology was used as a reference to define the pLOA in the relaxed GLUE LOA analysis using forcing and observational datasets from a single catchment. More similar case studies should be conducted on catchments with different hydrologic characteristics to assess the scope of this approach under different condition.