The study catchments are all located in the 106 000 km2 Tennessee River basin in the southeastern United States (Fig. 1), which is one of the most diverse temperate freshwater ecosystems in the world (Abell et al., 2000). A large number of endemic fish species and a unique assemblage of mussels, crayfish, and salamanders make the Tennessee River basin an excellent area for ecohydrological studies (Abell et al., 2000). From a study published by Knight et al. (2008), 25 catchments in the Tennessee River basin with observed streamflow time series (U.S. Geological Survey, 2016b), precipitation (U.S. Department of Commerce, 2007a), temperature (U.S. Department of Commerce, 2007b), and potential evaporation data (Rotstayn et al., 2006) were selected. The catchment areas range between 100 and 4800 km2 with elevations ranging from 174 to 937 m (U.S. Geological Survey, 2016a). Land cover for the study catchments is predominantly hardwood forest and pasture. Air temperature and precipitation vary between catchments according to both catchment elevation and longitude. Mean annual air temperature in the 25 catchments varies between 9.3 and 14.7 ∘C, and annual precipitation varies from 1500 to 2020 mm, with fall being slightly drier and less than 8 % of annual precipitation falling as snow. Runoff is highest in winter and lowest in summer, ranging from 400 to 1300 mma-1 (millimeters per year). Variability in soil thickness (Omernik, 1987), regolith thickness, karst development, and topographic slope (Hoos, 1990; Wolfe et al., 1997; Law et al., 2009) are documented as asserting the most influence on runoff.

Thirteen SFCs assessed in this study were chosen for use in model scenarios based on discernible functional connections with fish community diversity (Knight et al., 2008, 2014). This set of 13 SFCs represents each of the major flow regime components commonly used in ecological studies (e.g., Olden and Poff, 2003; Arthington et al., 2006; Caldwell et al., 2015): magnitude, ratio, frequency, variability, and date (Table 1). For this study the SFCs were additionally grouped according to flow conditions (mean, low, and high flow), because different aspects of the hydrograph have been shown to be sensitive to the objective function used for model calibration (for an overview, see Pfannerstill et al., 2014). The SFCs were calculated using the U.S. Geological Survey (2014) EflowStats R package. Please note that some of the tested SFCs (DH13, ML20, MA26, DH16, and FL2) are defined as scaled with the median, mean, or total runoff. The scaling leads to SFC values that are dependent on flow magnitudes. The magnitude of the simulation error for DH13, ML29, MA26, DH16, and FL2 is therefore dependent on runoff magnitudes, whereas the sign of the simulation error is not affected by the normalization.

The HBV (Hydrologiska Byråns Vattenavdelning) model (Bergström, 1976; Lindström et al., 1997) is a bucket-type hydrologic model for simulating continuous runoff series. Model inputs are daily rainfall and air temperature, as well as daily potential evaporation values. Hydrologic processes are represented by four different routines corresponding to snow, soil water, groundwater, and runoff routing, with a combined total of 16 parameters. In the snow routine, snow accumulation and snowmelt are calculated by a degree-day method. Snowmelt together with rainfall and potential evaporation are input to the soil-water routine, where the actual evaporation and the groundwater recharge are computed based on the soil-moisture storage. The groundwater (or response) routine consists of a connected shallow and deep groundwater reservoir and simulates peak flow, intermediate runoff, and baseflow. These three runoff components are taken together and transformed by a triangular weighting function during the routing process to calculate the runoff at the catchment outlet. Runoff can be modeled in a semi-distributed way by separating a catchment into elevation bands. Thereby, the snow and soil-water routines are calculated for each elevation band, whereas the groundwater storage and the runoff routing routines are treated as a lumped representation of the entire catchment. HBV exists in different versions, whereby the general structure of the model remains the same. The version applied in this study is HBV-light (Seibert and Vis, 2012). Like for all bucket-type models, parameters in the HBV model cannot be determined a priori: they are identified by model calibration instead. More detailed information on the HBV model can be found in Bergström (1976), Lindström et al. (1997), and Seibert and Vis (2012).

Figure 6a shows simulation results for the objective function ISingle for all 25 catchments and both modeling time periods. The five SFCs with the highest robustness (less variability in error; Fig. 6a) were RA7, ML20, FH6, E85, and MA41. All five of these SFCs could be used for the objective function IMulti; however, E85 (lowest 15 % of daily runoff) was discarded as potential SFC for IMulti because of its redundant information with ML20 (baseflow). The information value of the remaining 4 SFCs for each of the 13 SFCs is presented in Fig. 6b. All 13 SFCs were relatively well simulated by model calibrations with ISingle of either RA7, ML20, FH6, or MA41 (colored circles in Fig. 6b) compared to calibrations with other SFCs.

Median estimates of the 13 SFCs in the calibration period were slightly lower when the model was calibrated with IMulti rather than IMulti_Reff. Both of these objective functions led to better model performance for SFCs than calibrating with Reff alone (Fig. 3a). Model performance for the validation period with IMulti_Reff had a lower median error for SFCs than the error associated with using IMulti as an objective function (Fig. 3b). The comparison of IMulti and IMulti_Reff for all SFCs separately (Fig. 7a) revealed that for most SFCs both objective functions resulted in similar estimates. ISingle_Reff was better for estimating SFCs than IMulti_Reff, especially for SFCs not included in the IMulti_Reff objective function (Fig. 7b). Comparing simulations from IMulti_Reff and Reff revealed a smaller median error of the SFCs when calibrating with IMulti_Reff (Figs. 3b and 7c). Yet, for most SFCs not explicitly incorporated into the objective function IMulti_Reff, the objective function Reff performed equally well or slightly better than IMulti_Reff (Fig. 7c).

Figure 8 provides an overview (median of all 25 catchments) of how well SFCs were simulated by presenting the results for both modeling time periods and all five objective function types. Error magnitudes ranged between -25 and 25 % for the majority of SFCs. Considerably higher estimation accuracy was achieved for ML20 (-5 to 2 %), whereas estimation accuracies were lowest for MH10 and TL1, with error magnitudes up to 40 and 77 %, respectively. For some SFCs (e.g., MA26 and TL1) the error tended to be higher in one of the two modeling time periods, whereas for other SFCs (e.g., RA7 and MH10) the objective function had a distinct influence on the error magnitude. There was no evidence that the estimation accuracy depends on flow components (magnitude, ratio, frequency, variability, and date) or flow conditions (low, medium, and high flow).

The median error (illustrated by stars in Fig. 8) was used for the evaluation of the underestimation or overestimation of SFCs. Among the tested SFCs, an underestimation was observed for all five SFCs representing high-flow conditions as well as for three of four mean-flow-related SFCs. With one exception, low-flow SFCs were overestimated. This overall pattern was less evident when evaluating each objective function and time period separately (Figs. 8 and 9). The SFCs DH16 and MH10 indicate two typically observed deviations in the overall pattern. DH16 is an example of a SFC that could be regarded as being clearly underestimated by the model, because of its negative bias in 9 out of 10 cases (median values in Fig. 9a). However, for objective functions or modeling time periods with a low magnitude in the median bias, the underestimation of the SFC was not statistically significant. Even in the case of a median pointing to statistically significant underestimation, there might be a substantial number of catchments for which DH16 was overestimated. A second commonly observed phenomenon is shown by the SFC MH10 (Fig. 9b). While MH10 had mostly small but statistically significant median errors, there were many catchments with considerably higher errors. Although MH10 was the most extreme example, it illustrates that small median errors do not guarantee good results for all catchments.

The results demonstrated that the objective function used for model calibration strongly influences the estimation accuracy of SFCs. This finding confirms the findings of previous studies (e.g., Hingray et al., 2010; Westerberg et al., 2011; Murphy et al., 2013; Olsen et al., 2013; Pfannerstill et al., 2014; Shrestha et al., 2014; Caldwell et al., 2015; Vis et al., 2015) and points out the importance of making a careful choice of the objective function for model calibration. The benefit of optimizing one specific SFC lies in the relatively accurate estimation of the respective SFC compared to a calibration with Reff or a multi-SFC objective function. Model calibration on one single SFC clearly emphasizes the hydrograph aspects of the selected SFC possibly neglecting an adequate representation of other hydrograph characteristics. This implies that calibrations with ISingle can lead to poor model performance for SFCs not included in the objective function. The fact that a calibration with Reff and a calibration with multiple SFCs lead to comparable estimates for most SFCs indicates that the main hydrological processes of the catchments are similarly well represented with the two approaches. Considering that SFCs not incorporated into the objective function IMulti showed little change compared to calibrations with Reff brings into question the benefit of including SFCs in model calibration instead of applying a traditional calibration approach when aiming at estimating a suite of SFCs. This is surprising because the SFCs selected for IMulti or IMulti_Reff provide information on high flows, recession rate, percentage of baseflow, and annual runoff volume, and therefore should help in constraining the model with respect to different important runoff processes. These results are different from those of Yilmaz et al. (2008) and Pfannerstill et al. (2014), whose multi-metric runoff model calibration resulted in an improved general shape of the hydrograph. Although their calibration approach was mainly based on various segments of the flow duration curve, it is unclear why the conclusions differ that much. From the above discussion it becomes evident that calibrating a runoff model for estimating many different SFCs from one single hydrograph is a trade-off between finding a parameterization that is general enough to represent different aspects of the hydrograph and that simultaneously emphasizes specific SFCs. These trade-off situations are common as perfect model parameterizations are usually not possible due to a variety of uncertainty sources, such as model structural uncertainty and input and runoff data uncertainty (Beven, 2016).

A noticeable result from the current study is the distinct difference in model performance in calibration and validation when using the objective function ISingle. While almost perfect fits are achieved in calibration for all catchments and SFCs, model errors tend to be much higher in validation, with a considerable spread between catchments as well as a clear difference depending on the SFC. This observation confirms that the model is able to simulate the SFCs well, but also outlines that a good model calibration does not imply robust simulations in validation. In general, it seems that SFCs that are strongly related to physical catchment properties (e.g., rate of streamflow recession) are the most robust, followed by SFCs representing an average flow condition with a moderate robustness. SFCs that are a measure of more extreme high-flow conditions are the least robust, possibly because these conditions are subject to inter-annual weather changes and are more difficult to model due to their dynamic behavior. A low robustness could also indicate that the model structure might be suboptimal for some catchments.

The two least robust SFCs are MH10 and TL1. MH10 simulations with ISingle yield by far the poorest results of all objective function types, with very large normalized error in both positive and negative directions. In comparison, the high estimation errors for TL1 depend on the modeling time period. The high estimation errors for TL1 in period 2 stem from years where the minimum runoff was simulated in late winter while the observed minimum was in late fall. By visually analyzing the temperature and runoff time series, it can be hypothesized that such model simulations mainly happened in years with successive weeks of continuously little precipitation during late winter. Such prolonged drier periods occurred more often in one of the two modeling time periods and thus evoked the distinct bias in model accuracy depending on the simulation period. Both TL1 and MH10 are calculated from a single value per year, as opposed to, e.g., RA7, which is based on all recessions. In model calibration, many parameter sets are derived that perfectly simulate this single value. However, a good simulation of either TL1 or MH10 is not so much dependent on an accurate representation of dominant runoff processes. Thus, model results for the validation period using input data of identical quality can fail to accurately simulate either SFC because of parameter sets “tuned” to the data as opposed to being based on modeling the process.

The runoff model tends to underestimate SFCs related to mean and high-flow conditions, while SFCs representing low-flow conditions are generally overestimated. These results are consistent with those of Olsen et al. (2013), Caldwell et al. (2015), Vis et al. (2015), and Kiesel et al. (2017) and can partly be explained by the model behavior characterized by a less pronounced runoff response to precipitation events but increased groundwater discharge to the stream during drier periods compared to the observed data (Vis et al., 2015). The observations that average flow conditions are better simulated than extremes (Caldwell et al., 2015; Vis et al., 2015) or that high-flow-related SFCs are more accurately estimated than those related to low flow (Shrestha et al., 2014; Ryo et al., 2015) cannot be confirmed with our results. None of these earlier studies explicitly included SFCs in model calibration and the deviating results could be attributed to the differing approaches to defining the objective function(s). This presumption is supported by the previously described differences in results of Vis et al. (2015), although they applied the same runoff model, catchments, and SFCs.

The current study supports the assumption that including SFCs in model calibration helps to preserve most hydrograph aspects relevant to those SFCs. Thus, an objective function based on several SFCs is expected to result in a hydrograph from which a suite of SFCs can be calculated. Not knowing which SFCs will be relevant for a given study, a guideline as to which SFCs the model calibration could be based on would be helpful. The first step towards a guideline consists of selecting SFCs that are potentially valuable for model calibration. This selection was based on the concept of robustness and information value of SFCs, which is comparable to the approach used by Euser et al. (2013), who assessed the realism of model structures. Like Euser et al. (2013), results from the current study indicated that high robustness was not necessarily related to high information value, emphasizing the importance of selecting SFCs by jointly evaluating robustness and information value. The concept of information value and robustness favors simulations that preserve important hydrograph characteristics, as can be seen from the slightly improved median estimation accuracy of SFCs with the objective functions IMulti or IMulti_Reff compared to estimations with Reff only.

A model calibrated on certain flow conditions (low, medium, and high flow) is beneficial for SFCs representing these flow conditions (see, e.g., Murphy et al., 2013), so it was hypothesized that the information value of the selected SFCs is highest for SFCs belonging to the same group of flow conditions. The confirmation of this hypothesis would allow us to draw general conclusions about a minimum number of SFCs required for model calibration. Surprisingly the results did not reveal any pattern related to flow conditions and thus no recommendation for the final selection of SFCs can be made. It seems that the selection of SFCs for an informative and robust objective function depends on the type and the combination of SFCs one is interested in. Since this study was based on a limited number of SFCs it could be interesting to test the hypothesis by analyzing a greater number of SFCs. Testing a larger number of SFCs might reveal relations that are difficult to see with a small sample. Furthermore, more knowledge about the effect of single SFCs or the combination of SFCs used as objective functions on runoff simulations could be gained by using synthetic data and a modeling approach where an excellent hydrograph fit is possible (e.g., “HBV-land” in Seibert and Vis, 2012).

The emphasis of SFC-related modeling studies changed from estimating single SFCs to simulating a suite of SFCs (Olden and Poff, 2003). The modeling design of this study combined both approaches for the same SFCs and catchments and thus enabled a direct comparison of the results. Ideally, the runoff model could be calibrated to simulate a hydrograph for each catchment from which any SFC can be calculated. Such an approach ensures a relatively small calibration effort, which is especially valuable if one is interested in modeling many catchments and/or various scenarios. However, results indicate that SFCs related to a more generally calibrated model (e.g., Reff, IMulti, or IMulti_Reff) are less accurate than when they are estimated from hydrographs based on targeted model calibrations (e.g., ISingle or ISingle_Reff). This fact has substantial implications for the later application of simulated SFCs in decision-support systems for integrated resource management. As stated by Carlisle et al. (2010), with high errors in SFC estimates, only considerable flow departures from natural conditions can be detected. Also, inaccurate SFC values can impede the generation of more robust flow alteration–ecosystem change relationships that are ultimately needed for sustainable flow management guidelines (Arthington et al., 2006; Poff and Zimmermann, 2010; Gillespie et al., 2015; Cartwright et al., 2017).

As with regional statistical approaches, incorporating SFCs into model objective functions implies that a modeler knows which SFCs are relevant and that the model must be recalibrated if one is interested in additional SFCs. The advantage of runoff models over multivariate regressions and observed streamflow series includes their use for climate scenario analysis or for simulating runoff in ungauged catchments, with the latter being one of the ultimate aims in the ELOHA framework (Poff et al., 2010). Modeling SFCs gets even more challenging when moving from a gauged to an ungauged catchment. An appropriate calibration strategy targeted to the main simulation goal is crucial for any subsequent regionalization.

When comparing SFCs estimated from simulations of different runoff models, the question can be raised whether the results depend on the selected model. This question is especially important for resource managers who need to make decisions based on model results from different studies (Caldwell et al., 2015). A comparison of runoff models with different spatial scales that rely on different data inputs was conducted by Caldwell et al. (2015). Their results do not indicate that a certain runoff model is more suited for predicting SFCs than others, but rather that the calibration process probably has as much influence as the model structure. Thus, it can be assumed that the conclusions of this study would be similar if a different calibrated runoff model was applied.