Computer models of hydrologic systems are frequently used to investigate the
hydrologic response of land-cover change. If the modeling results are used to
inform resource-management decisions, then providing robust estimates of
uncertainty in the simulated response is an important consideration. Here we
examine the importance of parameterization, a necessarily subjective process,
on uncertainty estimates of the simulated hydrologic response of land-cover
change. Specifically, we applied the soil water assessment tool (SWAT) model
to a 1.4 km

An important use for computer models of hydrologic systems is simulation of
the hydrologic response of land-cover change

Previous research has shown that the subjective process of selecting which
model inputs to treat as uncertain (e.g., parameterization) may affect
uncertainty estimates in model outcomes

Woody-plant encroachment into grasslands has been a worldwide phenomena in
the past 150 years

Many hydrologic modeling analyses have been completed to evaluate the
feasibility of applying brush management in order to decrease the quantity of
water transpired within a given watershed.

To demonstrate the utility of including uncertainty estimation and to
investigate how parameterization may affect the reliability of a model to
resolve the hydrologic outcomes of simulated land-cover changes, such as
brush management, the SWAT

Study area and watershed location. The 47 HRUs yielded by the
ArcSWAT tool

The brush-management simulation described herein is applied to a 1.4 km

According to

The SWAT model was used to simulate the hydrologic response of the watershed,
including the effects of brush management. Specifically, a SWAT2012

Three datasets were needed to apply the ArcSWAT tool

the 10 m National Elevation Dataset (NED)

the Soil Survey Geographic Database (SSURGO)

the National Land Cover Database (NLCD)

These three datasets were used within the ArcSWAT tool to find unique land slope, soil, and land-cover combinations across the watershed. These unique combinations became HRUs in the SWAT model. The NED digital elevation model for the watershed was smoothed with a 4-pixel-width averaging kernel to remove apparent artifacts.

Summary of

As part of the previous study that evaluated the effects of brush management
at the Honey Creek State Natural Area

The ArcSWAT tool

The NLCD 2001

The application of the ArcSWAT tool with the previously described datasets
resulted in a model with a single subbasin covering the 1.4 km

The modeling analysis described herein includes two specific simulation
periods that correspond to the pretreatment and posttreatment
configurations:

1 January 2002 to 31 December 2003 (pretreatment configuration)

1 January 2005 to 31 December 2010 (posttreatment configuration).

Summary of parameters used in the reduced parameterization. These 12
inputs were selected from Table 1 in

The conditioning period and forecast period models simulate the years 2001 and 2004, respectively; the initial year of simulation for each model is used as a model warm-up period to remove any transient artifacts from initial conditions.

In a typical modeling feasibility study, the model is constructed and conditioned to pretreatment (conditioning period) system states, then forecasts are made using the model related to how simulated brush management will affect the hydrology within the watershed.

Here, two distinct SWAT models were constructed. The first SWAT model
simulated the pretreatment configuration and is hereinafter referred to as
the “pretreatment” model. The second SWAT model simulated the
posttreatment configuration and is hereinafter referred to as the
“posttreatment” model. The only difference between the two SWAT models are
specific inputs to HRUs 18, 20, 22, and 32, which represented the area of
watershed that was converted from evergreen forest (e.g., ashe juniper) to
rangeland. Modifications to the input files for the listed HRUs were as follows (herein,
references to specific SWAT input variables are shown in all caps):

– the CANMX variable in the .HRU input files

– the PLANT_ID and HEAT_UNITS variables in the .MGT input files.

In this study, brush management is simulated by modifying the maximum canopy
storage and inputs that control the simulated growth cycle for a
representative area of the subbasin from evergreen forest to rangeland,
because this required few assumptions and allowed injection of the desired
uncertainty into the simulation workflow. We modified the maximum canopy
storage and the plant growth aspects of HRUs 18, 20, 22, and 32 because these
inputs directly affect the available precipitation for partitioning and
simulated ET processes, respectively, whereas plant-growth variables affect
the timing and intensity of simulated ET processes related to the annual
plant-growth cycle. In the pretreatment model, these model inputs were
specified to represent ashe juniper land cover for HRUs 18, 20, 22, and 32,
whereas in the posttreatment model, these inputs for HRUs 18, 20, 22, and 32
were specified to represent rangeland land cover, effectively capturing the
change in the simulated inputs that corresponds to the brush-management
operations that occurred during 2004. See the SWAT theory

Parameterization is a critical part of any modeling analysis and has received
considerable attention in the literature

uses the 12 model inputs listed on Table 1
of

used 1305 model inputs. It builds on the 12 parameters
of the reduced parameterization by adding unique multiplier parameters on the
HRU scale for each of the 12 parameters in Table

These two parameterizations represent different approaches to hydrologic modeling. From a computational standpoint, the reduced parameterization is more desirable, whereas the full parameterization offers the opportunity for a more complete expression of model input uncertainty.

The SWAT input CANMX is of particular importance in simulating brush
management because it controls how much precipitation is available for
partitioning, and it is directly affected by land-cover changes. Therefore,
CANMX potentially exhibits a strong control of the simulated outcomes of
brush management. CANMX is not treated as uncertain in the reduced
parameterization as it is not commonly treated as adjustable

the parameter

the parameter

the parameter

In this way, we can incorporate uncertainty in the values of CANMX for all
three land-cover types while also enforcing the relations we expect for the
maximum canopy storage between the land-cover types. This treatment for CANMX
allows both the pre- and posttreatment models to receive the same parameter
values for the same land-cover types. Because HRUs 18, 20, 22, and 32 switch
from evergreen land cover to rangeland land cover, the CANMX values assigned
to these HRUs is in harmony with the CANMX values assigned to other HRUs. The
HRU-scale multipliers, named

The upper and lower bounds of each parameter were defined using a combination
of literature values

Both the pre- and posttreatment SWAT models must be evaluated repeatedly to simulate hydrologic outcomes of brush management and evaluate the importance of parameterization in said outcomes. To accomplish this repeated evaluation, a model-independent interface to SWAT was constructed. This interface facilitated the translation of parameter values into SWAT model input files, the execution of both the pre- and posttreatment SWAT models, and the postprocessing of SWAT model output into quantities of interest.

To translate parameter values to SWAT model input files, parameters were
assigned two characteristics:

Scale: a given parameter is either subbasin-scale or HRU-scale. Subbasin-scale parameters are applied to all 47 HRUs, whereas an HRU-scale parameter applies only to a specific HRU.

Type: a given parameter is either a multiplier-type parameter or a value-type parameter. Multiplier-type parameters are treated as scaling factors against the original SWAT model input variable(s), whereas value-type parameters replace the original SWAT model input variables(s).

The following steps represent a single model evaluation in the model
interface:

Construct two“base” tables of HRU-scale inputs where the columns are the SWAT model input names and the rows are the 47 HRUs (one table for the pretreatment model and one table for the posttreatment model). Populate these tables with the base input values from the ArcSWAT tool.

For each value-type, subbasin-scale parameter, replace the values in the base tables for each corresponding column with the specified parameter value, assigning all HRUs the same value.

For each multiplier-type, subbasin-scale parameter, multiply the corresponding column of the base tables by the specified parameter value, scaling all HRUs by the same value.

Apply

For each multiplier-type, HRU-scale parameter, multiply the corresponding row–column location in the base tables by the specified parameter value, scaling only a single entry in the table.

Translate the base tables into the appropriate SWAT input files for both the pre- and posttreatment models.

Apply precipitation multiplier parameters and write a new SWAT .PCP input
file

Apply plant-growth multiplier parameters and write a new SWAT plant-growth database file.

Run the pretreatment model for 2001 through 2010 (the pretreatment model outputs are needed from 2005–2010 for calculation of brush-management quantities of interest).

Run the posttreatment model for 2004 through 2010.

Postprocess both model runs to formulate brush-management quantities of interest and conditioning measures (described in Sect. 2.6).

The forward run process was completed many times as part of both the global sensitivity analysis and the uncertainty analysis (described in Sect. 2.6). For the reduced parameterization, the HRU-scale parameters, precipitation parameters, and plant growth parameters were each assigned a value of 1.0, effectively removing the influence of these parameters on the model outputs.

We used uncertainty quantification techniques to investigate how well the
previously described SWAT models simulate the effects of brush management on
long-term water-budget components. Specifically, after applying the global
sensitivity analysis (GSA) method of Morris

Output from both the pre- and posttreatment model was processed into QOIs
that encompass the simulated pre- and posttreatment long-term water-budget
components in the simulated watershed:

volumetric conditioning-period (pretreatment) ET–precipitation ratio

volumetric conditioning-period (pretreatment) streamflow–precipitation ratio

volumetric forecast-period (posttreatment) ET–precipitation ratio

volumetric forecast-period (posttreatment) streamflow–precipitation ratio

volumetric forecast-period difference between the simulated treated and untreated watershed.

The work of

QOI-5 is the primary quantity we use to evaluate the effectiveness of brush management: how does the simulated long-term volumetric ET change as a result of brush management? QOI-5 is simulated by running the pre- and posttreatment models for 2004 to 2010 and summing the differences in simulated ET between the two simulations.

Monte Carlo analysis

To perform the MC analysis, a 1 million parameter set ensemble was drawn
from the Prior for each of 1305 elements of the full parameterization
using the python module pyEMU

The reduced parameterization was evaluated in a similar fashion. The full-parameterization prior ensemble was modified so that the value of each parameter that was not included in reduced parameterization was fixed at the value representing the midpoint of the parameter's range. In this way, parameters not included in the full parameterization were treated as if they were not in the analysis and are instead “fixed” or “known” model inputs – just as they would be treated in a modeling analysis that only adjusted the 12 inputs of the reduced parameterization. Whereas the midpoint values of the fixed parameters may not be “best” in the sense that they reduce model-to-measurement misfit, they are nonetheless centered within the range of plausibility as described by the Prior.

The reduced-parameterization prior ensemble was also evaluated using the SWEEP utility in a distributed parallel environment, yielding 1 million values for each of the conditioning measures and brush-management QOIs.

Once the prior ensembles of both the reduced and full parameterizations were
evaluated, the GLUE method of

conditioning-period (pretreatment) Nash–Sutcliffe model efficiency coefficient (NSE)

conditioning-period (pretreatment) percent bias

conditioning-period (pretreatment) coefficient of determination
(R

These conditioning measures are widely used to judge a hydrologic model's
ability to reproduce observed daily mean streamflow

Realizations in each of the prior ensembles that satisfied all three of conditioning measures are designated as “behavioral” and, taken together, comprise the reduced and full parameterization behavioral ensembles, respectively. These behavioral ensembles represent parameter realizations that respect the Prior but that also reproduce daily mean streamflow acceptably well according to the three conditioning measures. That is, each parameter realization in the full- and reduced-parameterization behavioral ensembles can be considered “calibrated” in that each of these parameter realizations results in simulated daily mean streamflow that acceptably matches the observed data according to the three conditioning measures.

Given the large difference in the number of parameters between the reduced
(12) and full (1305) parameterizations, the interested reader may be
wondering how many members of the reduced and full parameterizations
influence either the conditioning measures or the QOIs or both. In an effort
to address this question, we employed the method of Morris

The application of the method of

Of the 1305 model inputs treated as parameters, the method of Morris analysis
indicates that only 194 parameters are noninfluential to the three conditioning
measures and five brush-management QOIs (see the Supplement for a complete
summary of the GSA results, including a table of the five most influential
parameters for each QOI and conditioning measure, Tables S2 and S3). Note
that many of the most influential parameters, specifically precipitation
multipliers, plant growth parameters, and HRU-scale parameters, are not in
the reduced parameterization and are not included in typical hydrologic
modeling analyses

Values of conditioning measures for the full (gray) and reduced
(blue) parameterizations. The diagonal panes

The MC–GLUE analysis yielded 7155 and 6846 realizations (out of the 1 million
member prior ensembles) that compose the behavioral ensembles for the reduced
and full parameterizations, respectively. These behavioral ensembles
reproduce the pretreatment daily mean streamflow data acceptably well
according to the three conditioning measures. The relation of prior and
behavioral ensembles to the three conditioning measures for the reduced and
full parameterizations can be seen graphically in Fig.

In general, for both the reduced and full parameterizations, the behavioral
distributions for ET-based QOIs (QOI-1 and QOI-3) are similar to prior
distributions; conditioning has slightly shifted the distributions towards
larger precipitation–ET ratios but has not substantially decreased the width
of the distributions. The similarity between prior and behavioral
distributions indicates the conditioning process has not changed the
uncertainty that exists in model-simulated ET. The prior and behavioral
distributions of reduced and full parameterizations bracket the measured
value for QOI-1, QOI-2, and QOI-3 at the 95 % confidence level
(Figs.

QOIs related to streamflow (QOI-2 and QOI-4) have markedly different
behavioral distributions compared to prior distributions, indicating
considerable conditioning of streamflow-sensitive parameters. The measured
value for QOI-4 (volumetric forecast-period – posttreatment –
streamflow–precipitation ratio) was not bracketed at the 95 % confidence
level by either behavioral distribution or the prior distribution of the
reduced parameterization (Fig.

Quantity of interest QOI-1: simulated conditioning-period (pretreatment) ET as a percentage of precipitation. The prior and behavioral 95 % confidence intervals – defined by the confidence limits (CLs) – of both model parameterizations bracket the measured value. However, the conditioning process has little affect on uncertainty as the behavioral distribution is similar to the prior distribution.

Quantity of interest QOI-2: simulated conditioning-period (pretreatment) streamflow as a percentage of precipitation. The effects of the conditioning process can be seen as large reduction in the range of the behavioral distribution compared to the prior distribution. The prior and behavioral distributions for model parameterizations bracket the measured value.

Quantity of interest QOI-3: simulated forecast period (posttreatment) ET as a percentage of precipitation. All 95 % confidence intervals bracket the measured value. However, the conditioning process has done little to decrease uncertainty, as the behavioral distributions are similar to the prior distributions for both model parameterizations.

Quantity of interest QOI-4: simulated forecast period (posttreatment) streamflow as a percentage of precipitation. Both the parameterizations appear to have been “overfit” with respect to this QOI, as neither behavioral distributions bracket the measured value at the 95 % confidence level.

Quantity of interest QOI-5: simulated difference in total forecast period (posttreatment) ET volume as a result of brush management. Negative values indicate a decrease in ET as a result of brush management. The reduced parameterization yields a much narrower confidence interval compared to the full parameterization.

The prior uncertainty in the QOI-5 (the simulated difference between the
total forecast-period ET in the pre- and posttreatment models) was
substantially larger for the full parameterization compared to the reduced
parameterization (Fig.

QOI-5 behavioral uncertainty from the reduced parameterization is
substantially different than the prior uncertainty; the 95 % confidence
interval of the reduced parameterization behavioral distribution ranges from

The full-parameterization behavioral distribution of QOI-5 included a range
of possible outcomes from a net decrease to a slight net increase in the ET
component of the long-term water budget (Fig.

This study demonstrates the importance of robust uncertainty quantification to support simulations of brush management, and, more generally, simulation of the hydrologic outcomes of land-cover change. Without uncertainty quantification, the simulated outcomes of simulating brush management are simply single points on the behavioral QOI distributions, which conveys no information related to the reliability of the model results. The failure of the reduced-parameterization model to provide robust uncertainty estimates demonstrates the importance of parameterization when attempting to quantify uncertainty in land-cover change simulations. The results of our analysis should not be directly extrapolated to other hydrologic settings that are different from the one described herein.

The MC–GLUE analysis showed that using a reduced parameterization to represent model input uncertainty leads to a misrepresentation and critical underestimation of the uncertainty in QOI-5, leading to artificially high confidence that brush-management activities will decrease the ET component of the water budget by approximately 2.0 to 2.5 %. By including a more representative and complete set of parameters to represent model input uncertainty, the resulting QOI-5 uncertainty estimate more appropriately conveys the reliability in the modeled outcome of brush management.

A clear link between level of parameterization and uncertainty estimates for
the simulated results of brush management has been demonstrated, and issues
such as underestimation of uncertainty and numerical artifacts are shown to
be associated with a reduced parameterization. Furthermore, the results of
applying the method of Morris revealed more than 1100 model inputs that were
identified as uncertain and that also influence conditioning measures, QOIs
or both. Following

There are two avenues to reduce QOI-5 uncertainty: (1) collect information directly about the model input variables that most influence QOI-5 – that is, reduce the prior uncertainty of the parameters that represent these inputs – or (2) collect additional hydrologic observations that, through conditioning, reduce the uncertainty of parameters that influence QOI-5. We recognize that the ET observation data used to formulate QOI-1 could in fact be used as a condition measure. Given the similarity between QOI-1 and QOI-5, it is possible that the conditioning-period ET data could be used to further condition several parameters that influence QOI-5, thereby reducing the behavioral uncertainty of QOI-5. However, the conditioning-period ET data provide a valuable validation of the model's performance, and using these data as a conditioning measure would provide unique and atypical conditioning.

This study provided an analysis of the ability of a SWAT model to forecast how brush management affects the long-term water balance within a watershed. The analysis relies on measured streamflow and independently derived evapotranspiration estimates to condition the parameterized model inputs as well as provide a verification of the model's performance during the forecast period. The method of Morris was used to investigate model input influence on conditioning measures and brush-management quantities of interest. Following the method of Morris, Monte Carlo and GLUE analyses were used to estimate the uncertainty of brush-management QOIs for the reduced and full parameterization schemes.

Our analysis reveals the importance of robust uncertainty quantification when simulating the outcomes of brush management, especially as it relates to how the model is parameterized. Failure to specify a complete and encompassing parameterization is shown to lead to an underestimation of uncertainty in simulated brush-management outcomes, which may lead to suboptimal water-resource decision making.

Given the number of identified uncertain model inputs and the associated specified uncertainty in said inputs, the model-simulated change in the long-term ET in the watershed is largely uncertain and includes a range of possible outcomes from a net negative to a slightly net positive change in the long-term ET component of the water budget. The resulting uncertainty in one of the primary metrics of brush-management effectiveness underscores the importance of robust and conservative uncertainty quantification. Watersheds with different hydrologic response characteristics will obviously behave differently, but, if modeling is used to evaluate brush-management outcomes, robust uncertainty quantification is needed to place the model results in a representative context.

A data release that supports the analyses presented herein
is available at

an ESRI ArcMAP 10.2.2 project that includes the ArcSWAT version 2012.10.2.18 project used to create the base model

base SWAT2012 input files generated by the ArcSWAT tool

PEST++ interface files including python pre- and postprocessing scripts.

The comma-separated value files used in the reduced and full-parameterization
Monte Carlo analysis can be generated from the files provided in the data
release

SR and VS gathered datasets and applied the ArcSWAT tool to prepared the SWAT model input files with help from JB. JW subjected the ArcSWAT model input files to the global sensitivity analysis and combined Monte Carlo GLUE analysis. JW prepared the paper with contributions from all coauthors.

The authors declare that they have no conflict of interest.

Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the US Government.

The authors would like to recognize Kyle Douglas-Mankin, as well as additional reviewers, whose insightful comments improved the paper. Edited by: Nunzio Romano Reviewed by: John Doherty, Patrick Belmont, Tammo Steenhuis, and Lieke Melsen