While lakes only cover around 4 % of the Earth's land surface , their impact on the climate system is disproportionate to their coverage . Lakes exert their influence on different timescales. In the long term, down to the seasonal scale, they interact with the climate system through, e.g., their influence on the global carbon balance . They also provide more immediate feedback through mass and energy exchanges with the atmosphere. There is an array of processes, some of them interacting, that modulate the impact of lakes, working at different timescales .

Models of such real-world systems are necessarily simplified conceptualizations, making them tractable . Inferences drawn from models will be plagued by issues such as epistemic gaps, data uncertainties or even computational artifacts . Taking modeling uncertainties into account should be of fundamental importance, but they might not be considered due to lack of evaluation data, computational limitations or a worrying avoidance of the issue, among other things.

In order to represent the influence of lakes in the climate system, lake models are embedded into land-surface schemes that can in turn be coupled to regional or global climate models. These cascading systems are linked through their inputs and outputs, sometimes considering feedbacks . The uncertainties in the couplings, even when propagated through simple systems, can produce a wide range of potential outputs . These, and other uncertainties, make mimicking the hydrological system in coupled land-surface–climate models a practical challenge for current modeling systems . A first step in improving these systems would be to quantify the uncertainty of the linkages, and if possible reduce it.

In general, hydrological aspects of the climate system were effectively ignored in early modeling efforts or summarily represented . This was mostly due to computational limitations but also to limited process understanding . Up until the 1990s, most open-water surfaces were not resolved in climate models : only large lakes could be represented . These early conceptualizations are simplistic, viewing lakes as saturated soils with modified roughness and albedo or as slabs of water with no differentiated mixing , and ignore the internal thermal structure of lakes, which influences fluxes to the atmosphere .

Lakes are not inert masses but living systems. From the point of view of atmospheric feedbacks, ecosystem function is more than just ontologically relevant and is a controlling factor for heat exchange. Previous studies illustrate the feedback between phytoplankton and thermal structure, via light extinction modulation . Thermal stratification modulates oxygen concentrations and therefore ecosystem function . Paleological studies of lake ecosystems show they are highly sensitive to environmental change . Understanding energy feedbacks between lakes and the atmosphere, or at least estimating the associated uncertainties, is of central importance to diagnose the potential impacts of change. Accounting for these uncertainties could anchor the results of studies such as and who show how lakes impact regional climate and contribute to greenhouse gas emissions .

More than half the global lake area consists of small lakes , which might not be resolved on the typical scales of global or mesoscale models. Furthermore, the spatial patterns of mass and energy fluxes directly influence the evolution of the atmospheric boundary layer, thus compounding the issue : there is a distinct difference in both the timing and the magnitude of fluxes between open-water surfaces and the atmosphere compared to land . The magnitude of these fluxes can be a function of several factors, such as lake area and the latitude of the lake . The clarity of the lake seems to be the dominant factor .

With different albedo, heat capacity, and surface roughness compared to the surrounding land areas, lakes also provide more immediate feedback through transfer of heat and moisture exchanges with the atmosphere e.g.,. While some studies have performed direct measurements of latent and sensible turbulent heat fluxes from eddy-covariance systems over lakes and reservoirs e.g., these measurements can be difficult and expensive, and as such improved modeling approaches are necessary e.g.,.

The Canadian Small Lake Model CSLM;, a 1-D, deterministic, bulk mixed-layer model, was developed to integrate within the Canadian Land Surface Scheme CLASS;, which can in turn be coupled to regional climate models as well as large-scale hydrological models. CLASS resolves heterogeneity in the landscape using a mosaic approach where the CSLM acts as a tile, generating its own flux exchange with the atmosphere. Previous work with CSLM has demonstrated its ability to reproduce surface temperatures over a range of conditions within different lakes of the Experimental Lake Area . Evaluation of the model in terms of surface heat fluxes is generally lacking, however. In this paper we use observed micrometeorological flux data from a small lake (Landing Lake, 114.4∘ N, 62.5∘ W, surface area = 1.12 km2) in the Northwest Territories of Canada (Fig. ) to explore model performance and demonstrate the capacity of alternative methods of sensitivity analysis to identify the relative significance of model parameters and their impact on uncertainty in the simulation of lake–atmosphere energy exchange.

Below we briefly introduce the case-study application and the model and then present and discuss alternative methods for model sensitivity analysis, drawn from the PSUADE toolbox. The paper presents a comparative analysis of these alternative approaches and concludes with a summary of key findings and general discussion of the implications.

CSLM is a 1-D (with depth as the vertical coordinate axis), bulk mixed-layer model that outputs the temperature profile within the water column and sensible and latent heat at each time step. The surface boundary is set using atmospheric conditions while the boundary at the base of the lake is adiabatic. The model is forced at each time step with meteorological data. Using an initial temperature profile, the surface energy balance is solved at the boundary while the conductive and radiative heat flux is solved at each depth interval. From this heat flux and using the 1-D heat equation, the temperature profile of the lake is recalculated for the current time step. At this stage if there are any static instabilities in the temperature profile, mixing occurs when an integrated turbulent kinetic energy (TKE) approach is used. This generates a final temperature profile. The surface temperature from the profile is then used within the bulk aerodynamic formulas to calculate both sensible and latent heat fluxes. A complete description of the model can be found in .

Along with the initial temperature profile and standard meteorological forcing, the light extinction coefficient (Kd, m-1) for a given lake is also required. The light extinction coefficient, Kd (Table ), is a measure of how light in the visible spectrum attenuates through the water column; a measure of the transparency of the lake. Low values of Kd indicate a clearer lake where light can penetrate deep into the water column. Higher values of Kd indicate a more turbid lake where light attenuates much closer to the surface. The value of this parameter has direct implications for the temperature profile within a lake and thus both the sensible and latent heat fluxes. The mechanisms underlying the thermal structure are complex. From a purely mechanistic perspective, water clarity affects lake hydrodynamics , even under a 24 h period . The thermal structure further depends on lake morphometry and is compounded by biogeochemical processes, such as browning waters and ecosystem function . Within CSLM, shortwave extinction is exponential with depth following Beer's law, with Kd identified as the e-folding depth.

The turbulence subroutine in CSLM is used to determine the mixed layer depth; the depth over which active mixing occurs, homogenizing the temperature profile. This is the final step within the model before the temperature profile and fluxes are output. The change in mixed layer depth is calculated by assessing the change in TKE. This change is determined through the competition between energy input terms which act to increase the mixed layer depth and loss terms which act to decrease the depth of the mixed layer. Energy input terms include wind-driven stirring and buoyancy fluxes Fq, transport of TKE to the thermocline Fi which acts to erode it, and shear production at the base of the mixed layer Fs. Loss terms include energy dissipation Fd, entrainment of deeper water at the thermocline Fp and sinks of TKE within the mixed layer FL. In this scheme the modeled integrated TKE budget is used to determine the turbulence within a mixed layer of uniform properties and depth h . This is expressed as follows: ddt12hEs=h2dEsdt+Es2dhdt, where Es2 is the average TKE per unit mass. To solve the energy budget, the terms on the right-hand side are rewritten as the sum of relevant turbulent processes: h2dEsdt=Fq-Fd-Fi,Es2dhdt=Fi+Fs-Fp-FL.

This parameterization involves five empirical turbulent coefficients (Table ). With the exception of the entrainment term, Fp, the definition of each of the terms contains a constant empirical coefficient: cn for surface mechanical input, ce for TKE dissipation, cf for TKE transport to the thermocline, cs for shear production and cL for the sinks of TKE. Experiments detailed in yield a range of values for cn, cf, ce and cs. A set of consistent values was chosen by and have been used subsequently by and : cn=1.33, cs=0.2, ce=1.15 and cf=0.25. chose cL=0.235 based on experimental data.

Both turbulent mixing and Kd directly control the output of temperature and heat fluxes. In addition, there is also a considerable amount of uncertainty in both. In the case of the turbulent subroutine, the values of the empirical constants, while consistent, have never been investigated. The value of Kd, however, carries uncertainty due to limitations in its measurement both spatially and temporally. Although measurement technology has long been available , continuous measurements of Kd are, to the best of our knowledge, relatively scarce despite the relative affordability of cosine collectors, perhaps currently the most widespread technology; see for a recent application. Also, point measurements can be taken through more or less direct proxies, such as the Secchi disk depth and dissolved organic compound (DOC) concentrations . The technology is in fact evolving . Alas, despite the availability of measurement tools, Kd might sometimes be an afterthought, as in the case here presented, where it was determined from DOC concentrations.

When considering model-performance analysis, particularly in the case of complex models, it is important to note that one or more model parameters might exert more or less influence on one or more model outputs. Some of these parameters may be observable and/or measured while others may have dubious physical interpretation. How to specify model parameters is not a trivial issue . Over-parameterization, parameter interactions, erroneous evaluation data and computational errors are all causes of equifinality : different parameter-value combinations yielding nigh-indistinguishable results. Sensitivity analysis (SA) provides a way to mitigate the equifinality issue by identifying the parameters that dominate model performance. Unimportant parameters may be used to reduce dimensionality, palliating equifinality with minimal impact on performance . Better constraints on important parameters (e.g., through improved measurements) may result in uncertainty reduction.

There are many different approaches for SA; see and for a thorough discussion. in particular provide an exhaustive overview of the state-of-the-art. In general terms, SA methods can be classified as global and local. Local measures assess model response by varying one parameter at a time while global measures vary several parameters simultaneously. Local measures do not account for possible parameter interactions but are computationally lighter since, all other conditions being equal, they require fewer evaluations. The less demanding method is in fact differential SA, which uses partial derivatives or finite differences at a location – parameter-value combination – of interest.

If the location of interest within the parameter space is not known a priori, a common occurrence given the equifinality issue, then random-sampling global SA measures are preferred. However, this necessitates more model evaluations since instead of varying just one parameter or looking at a specific location they are based on exploring the entirety of the feasible parameter space. The generalized likelihood uncertainty estimation GLUE; is a global SA method that evolved from the work of and consists of randomly sampling the prior parameter space and evaluating the performance of the model at each random parameter-value vector, selecting behavioral (well-performing) vectors either after subjective thresholding or using measurement error as a splitting criteria: the limits-of-acceptability approach .

The projection of the multidimensional parameter-value vectors into a plane defined by one of said parameters and the corresponding performance (“dotty plots”) can then be used to define a one-dimensional frequency distribution that is indicative of the parametric sensitivity. Furthermore, an estimate of the uncertainty of model simulations can be obtained by weighing them according to the performance, deriving uncertainty bounds. The GLUE approach, however, is computationally inefficient, especially since strong information about prior parameter distributions is often unavailable and random uniform sampling required.

Computational performance of global SA can be improved using the design-of-experiment approach , which consists of two steps. First, while still random, sampling is designed to efficiently cover the prior parameter space, or is tailored specifically for a given global SA method. Second, variation in model performance is attributed to the variation of different parameters; see . The relative ranking of parameters and quantitative attribution of model–output variance might change depending on the sampling technique and global SA method .

developed the PSUADE (Problem Solving environment for Uncertainty Analysis and Design Exploration) package that provides a collection of tools to perform uncertainty quantification and sensitivity analysis. PSUADE has been used to produce technical reports related to the modeling of explosives , to the modeling of a two-dimensional interaction between soil and foundation structure , and to the modeling of an electrostatic microelectromechanical system switch . use PSUADE to produce a book chapter exploring uncertainty quantification for multiphysics applications.

All the listed applications only focus on a subset of the methods available in PSUADE. Similarly, in hydrological applications, most SA studies focus on a single method (see Table 3 in for an overview of recent applications), sometimes disregarding global SA in favor of local SA: e.g., most applications of PEST where the number of calibration parameters can be reduced using local SA.

The present study was based on a combination of three factors underlining its relevance: firstly, by building upon existing literature that stresses the importance of lake clarity in modeling heat transfers and evaluating against measured fluxes, as done by for large lakes.

Secondly, while the difficulty in finding adequate parameterizations for land-surface schemes has been recognized , and the importance of incorporating observational data has been underlined , little effective attention has been placed on uncertainty analysis in this kind of physical modeling. Regarding lakes, inroads have been made, but with respect to water quality .

Thirdly, PSUADE is a recently available tool that provides the mechanisms to perform the kind of exhaustive SA pioneered by that allows easy testing of different methods within a single package and hence provides more robust results. Furthermore, quantifying the uncertainty in the connecting fluxes of the different components of a modular system (in this case a land-surface scheme) should be one of the first steps, often not performed, in an overall uncertainty assessment. This paper also represents a start in that direction. Our concrete objectives were to find the following. a.

What were the parameters that dominated model performance, in terms of latent and sensible heat fluxes, evaluated with two different objective functions, the Nash–Sutcliffe Efficiency NSE; and the mean absolute error (MAE)?

The CSLM simulates heat-transfer fluxes – latent and sensible heat – at the boundary between lake surface and the atmosphere. The model was run with half-hourly forcings and the resulting simulations were temporally aggregated to evaluate against daily flux data, which were obtained through integration of hourly eddy-covariance measurements. The aggregation was necessary because of inherent limitations of the higher-frequency daily covariance data that were available for the period 12 June–18 October 2007. Two performance metrics were used for the evaluation, MAE and NSE: MAE=∑i=1nOi-Sin,NSE=1-∑i=1nOi-Si2∑i=1nOi-O‾2, where n is the number of time steps at which the model was evaluated, Oi was the observed value at time step i, Si was the simulated value at time step i, and O‾ is the mean of the observed values. The rationale behind the choice was to contrast the propensity of NSE to prioritize better-fitting high values , sometimes to the detriment of other ranges, by using MAE as a contrasting metric, less dependent on high-value fit. The entire period with available data was used for the computation of the performance measures. The metrics were computed after aggregating the hourly data to a daily time step.

MAE and NSE were the basis of all tested SA methods (Table ). The PSUADE package is a tool that assembles SA methods under a unified computational framework, thus facilitating exhaustive testing in the vein of , who tested the impact of sampling, in terms of frequency and technique, and different SA methods in the identification of sensitive parameters for a hydrological model. The general procedure for PSUADE is as follows: (a) generate random samples, (b) run the model and compute the performance for all the samples and (c) compute SA metrics from the obtained performances. The performance used was the average performance for the simulation of latent and sensible heat.

From a pragmatic point of view, the choice of SA method might depend on the computational requirements of the tested model. Different methods might require different numbers of simulations to achieve consistent results, as shown by . The number of simulations required is primarily a function of the complexity of the response surface, a factor difficult to control, but also depends on the sampling scheme and the theoretical basis of the SA method. Different schemes represent different ways of exploring the parameter space and some are better suited than others depending on the purpose .

The CSLM was sufficiently fast to run, less than a second for 128 days of half-hourly data, so that the number of simulations required by the different methods was not a limitation. Therefore we ran as many simulations as necessary to obtain consistent results (Table ). The choice of sampling method was based on those used by in their experiment.

Seven qualitative and two quantitative SA methods were tested to identify the sensitive parameters in CSLM. The parameters chosen for the test are the ones deemed a priori to have physical significance for the thermodynamic functioning of the lake. The range of the parameters was based on physically plausible values (Table ).

Finally, in order to broadly estimate the impact of parametric uncertainty on heat-flux simulation, the GLUE procedure was applied to CSLM for the parameters, and ranges, listed in Table . A total of 1 million simulations were performed and the top 10 % selected as behavioral in order to compute the uncertainty bounds. For MAE, the lower the value, the better the performance, and therefore the inverse of the computed MAE was used as the likelihood when performing GLUE. The NSE values were used unmodified for the weighting.

Most SA methods pinpointed Kd as the most sensitive parameter in terms of simulating latent and sensible heat fluxes. The exceptions were those which had the strongest assumptions with respect to the nature of model response: SPEAR and SRC assume linearity between input (model parameter) and response (performance measure), which was not the case for CSLM. Somewhat surprisingly, GP, which also makes assumptions about the shape of the response surface and considers it Gaussian, also showed Kd to be the most sensitive parameter. This was either a false positive or the response surface might have been locally linear, thus justifying the normality assumption . None of the other methods makes such strong assumptions about the nature of the response surface, although localized fitting does occur: e.g., stepwise linear for MARS.

This predominance of Kd is perhaps not surprising given the recognized importance of the light extinction coefficient in modulating heat transfers . Given the complex and intertwined processes that can affect light penetration, such as browning waters and ecosystem function , a single measurement of its value might prove insufficient. It might be necessary to rely on continuous measurement in order to improve modeling, either through adherence to a parsimony principle (its value need not be modeled if actually measured) or stemming from the need to evaluate the complex processes influencing its value.

The light extinction coefficient is the only parameter considered in this analysis that is directly measurable (Table ). Being able to measure the most sensitive parameter is a definite advantage: it allows the number of parameters needing calibration to be confidently reduced. Furthermore, since Kd is so predominant in terms of model performance that making it a measured instead of a calibrated quantity should facilitate further model evaluation, whatever variability in the performance that was not a function of Kd should become easier to quantify and analyze since it would not be obfuscated by Kd's predominance.

With respect to large-scale applications, such as climate modeling and land-surface schemes, measuring Kd for every lake might be a practical impossibility, but its importance should be stressed and further research devoted to assess its temporal and spatial variability. Remote sensing might provide a solution to lack of in situ measurements and is in fact used to provide estimates of the Secchi depth , a proxy for the light extinction coefficient.

The model did not perform equally well at different time periods (Fig. ) and Kd is known to vary over time . A first step into assessing causality between these two factors would be to continually measure Kd, at least at a daily time step. Such measurements have proved useful in evaluating turbulent transfers over large lakes .

There is often a disconnect between experimentalists and computer modelers . The conceptual framework behind the present study is a testament to an improvement of those dialogues: the undertaken modeling approach was based on data from an observation station that was explicitly established to support testing of hydrometeorological models over lakes. Such new data facilitated the analysis performed here and it must be stressed that measuring Kd was not part of the objectives of the field campaign. The modeling exercise performed here underlined its importance for the simulation of heat fluxes and represents an argument in favor of its monitoring.

The limitations of SA methods are tightly related to the curse of dimensionality . All SA methods obviate the issue by randomly sampling the parameter space, implicitly assuming results to be representative of the entire space. This is especially problematic in high-dimensional spaces, where even the simplest of models might face prohibitive computational costs and even the most sophisticated sampling schemes are limited in their coverage of the parameter space.

The uncertainty bounds obtained from the GLUE analysis are nothing but subjective since the behavioral threshold was arbitrarily set for the top 10 % simulations. Even with this rather lenient criteria, the bounds did not encompass all observations, especially for sensible heat (Fig. ). This might be due to the tradeoffs between latent and sensible heat simulation (Fig. ) and the chosen performance measures, but might also stem from inadequate process representation. The clear tradeoffs in performance for latent and sensible heat might be influenced by the fact that while the CSLM surface energy balance is a strongly nonlinear function of the surface skin temperature, both the sensible and latent heat fluxes are linear terms in this relationship. All other things being equal, this leads to a direct tradeoff between them: the capacity of the model to simulate one of the terms is inversely proportional to its ability to simulate the other; see Fig. .

PSUADE is a powerful package that facilitates SA by providing a wealth of approaches and presents an opportunity that evaluate their appropriateness, a factor of special importance since they are based on different conceptual precepts. Overall the results might depend on the quality of the sampling, but that was not a factor here since the model was lightweight enough for the number of runs not to be a limiting factor. This will definitely not be the case with more complex models. If runtime is an issue methods like GLUE become nonviable. FAST, however, requires relatively fewer simulations to reach consistent results, but makes assumptions about the nature of the response surface. The latter is true for most RSM methods: the robustness of the results depends on the validity of these assumptions.

As such, nonparametric methods like McKay or MOAT might be preferable, the cost being large computational requirements. In any case the shakiest conceptual basis is for the methods assuming linearity in the model response, i.e., SPEAR and SRC, as that is seldom the case for environmental models. The method that provides the most information about parameter interaction is the SOBOL method, the cost being numerical instabilities, especially when a large number of parameters is involved. All methods might fail in the presence of singularities or discontinuities in the response surface and if they give the right answer it might be for the wrong reasons.

In all, to recommend a preferred method is difficult and very much depends on the nature of the problem. If the popularity of their use is an indicator, then the MOAT and RSM approaches are the most used in the literature . It is the authors' opinion that the SOBOL method provides the most complete insight but is difficult to apply in high-dimensional spaces and is prone to numerical instabilities.

Despite the shortcomings, the answer to our original question was clear: Kd is undeniably the most sensitive parameter in the simulation of heat fluxes.