Exploring the impact of forcing error characteristics on physically based snow simulations within a global sensitivity analysis framework

Raleigh, M. S.; Lundquist, J. D.; Clark, M. P.

doi:https://doi.org/10.5194/hess-19-3153-2015

Articles | Volume 19, issue 7

https://doi.org/10.5194/hess-19-3153-2015

© Author(s) 2015. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/hess-19-3153-2015

© Author(s) 2015. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 19, issue 7

Research article

|

20 Jul 2015

Research article |

| 20 Jul 2015

Exploring the impact of forcing error characteristics on physically based snow simulations within a global sensitivity analysis framework

M. S. Raleigh, J. D. Lundquist, and M. P. Clark

Download

Final revised paper (published on 20 Jul 2015)
Preprint (discussion started on 16 Dec 2014)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC C5846: 'Review', Francesca Pianosi, 30 Dec 2014
- AC C6748: 'Authors' responses to comments from Francesca Pianosi', Mark Raleigh, 15 Apr 2015
RC C5860: 'Interactive comment on “Exploring the impact of forcing error characteristics on physically based snow simulations within a global sensitivity analysis framework”', Jianduo Li, 31 Dec 2014
- AC C6761: 'Authors' responses to comments from Jianduo Li', Mark Raleigh, 15 Apr 2015
RC C5904: 'Review Hess-2014-461 “Exploring the impact of forcing error characteristics on physically based snow simulations within a global sensitivity analysis framework”', Anonymous Referee #3, 05 Jan 2015
- AC C6766: 'Authors' responses to comments from Anonymous Referee #3', Mark Raleigh, 15 Apr 2015
RC C5972: 'Review of hess-2014-461', Richard L.H. Essery, 08 Jan 2015
- AC C6779: 'Authors' responses to comments from Richard L.H. Essery', Mark Raleigh, 15 Apr 2015
RC C6152: 'Review comments', Rafael Rosolem, 21 Jan 2015
- AC C6793: 'Authors' responses to comments from Rafael Rosolem', Mark Raleigh, 15 Apr 2015
RC C6155: 'Interactive comment on “Exploring the impact of forcing error characteristics on physically based snow simulations within a global sensitivity analysis framework” by M. S. Raleigh et al.', Adam Winstral, 22 Jan 2015
- AC C6799: 'Authors' responses to comments from Adam Winstral', Mark Raleigh, 16 Apr 2015

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (24 Apr 2015) by Ross Woods

AR by Mark Raleigh on behalf of the Authors (20 May 2015) Author's response Manuscript

ED: Referee Nomination & Report Request started (21 May 2015) by Ross Woods

RR by Richard L.H. Essery (31 May 2015)

RR by Francesca Pianosi (05 Jun 2015)

Suggestions for revision or reasons for rejection

The paper has been substantially revised after the first round of reviews and I think it is significantly improved. In particular, with respect to the comments raised in my review, the discussion of GSA results and their interpretation is much more solid and convincing.

The authors found some inadequacies in their code implementing the error models and after fixing them they were able to produce GSA results that look much more reliable. They also enriched their discussion of GSA results with more insights on their physical meaning and implications, and more interpretation of ``surprising'' or unexpected results.

In my opinion this revised manuscript is suitable for publication and could constitute an interesting contribution to the readership of HESS, and a nice demonstration of how GSA methods can be used to learn about the behavior of a hydrological model and its sources of uncertainty.

I have few minor suggestions (mainly again relating to methodological aspects and references of GSA), that the authors may consider in a final revision of their manuscript.

Lines 376-378: ``Sensitivity analyses tend to focus more on the model parameter ... than on the forcing matrix''. True, however recently there have been an increasing number of applications considering other input factors than parameters, for instance

Baroni, G., Tarantola, S., 2014. A general probabilistic framework for uncertainty and global sensitivity analysis of deterministic models: A hydrological case study. Environmental Modelling \& Software 51, 26–34.

and references therein. It would be good to cite some of these works, also to stress that GSA does not need to focus on parameters only.

Lines 387-389: ``Sobol' sensitivity analysis uses variance decomposition to attribute output variance to input variance''. I would say ``to input variability'' or ``to input uncertainty'' (see similar comment in my previous review)

Lines 396-399: ``A key assumption to the Sobol' approach is that the factors are independent'': I know the authors introduced this sentence to address one of my previous comments, however the sentence is not technically completely correct (I was a bit sloppy about this in my review too). There are some recent papers that show how to apply the Sobol' approach in the case of correlated/dependent inputs (Lilburne &Tarantola, 2009. Sensitivity analysis of spatial models.; Kucherenko et al, 2012. Estimation of global sensitivity indices for models with dependent variables; Castaing and Gratiet, 2015, ANOVA decomposition of conditional Gaussian processes for sensitivity analysis with dependent inputs), however they require a sampling and approximation strategy different from the one implemented in this paper and described in Sec. 3.3.3. So maybe better rephrase the sentence like:
``A key assumption to the implementation of the Sobol' approach used in this paper (see Sec. 3.3.3) is that the factors are independent''

Line 436: ``Final S_{Ti} values'': what does ``final'' mean here?

Line 442 and following (and Figure 2): I think the term ``Sobol' values'' is a bit confusing, one could think it refers to values of the sensitivity indices rather than input samples, or suggest that there is a link between the quasi-random sampling strategy and the variance decomposition idea, while there is not (apart from the fact that Sobol' contributed to both). Maybe better use ``input factor samples'' or ``parameter samples''?

Lines 679-680: ``we calculated daily sensitivity indices of modeled SWE to forcing biases at each site and scenario (Fig. 8).'' I understand that here the output variable whose sensitivity is being assessed is the simulated SWE on given day (and sensitivity index is re-estimated at each day in the simulation horizon). If so, this is conceptually quite different from the output definitions used so far (aggregate indicators from the entire time series of simulated variables). Maybe good to add one sentence to further clarify the point? Readers not particularly familiar with GSA may struggle here.

Table 4: I understand the number outside parenthesis is the number of ``behavioural'' samples in the initial Sobol' matrix and the one in parenthesis is model simulations, the relation being the usual $N_{sim} = N_{samp}\times (k+2)$ where $k=8$ in your case study - but it took a while to figure out - maybe better just say it in the caption? Also, this seems to suggest that you checked output samples associated to the initial Sobol' matrix only, while I suppose you should also check the feasibility after resampling - in other words, check the final matrix $A_B$?

Hide

ED: Publish subject to minor revisions (Editor review) (10 Jun 2015) by Ross Woods

AR by Mark Raleigh on behalf of the Authors (17 Jun 2015) Author's response Manuscript

ED: Publish as is (18 Jun 2015) by Ross Woods

AR by Mark Raleigh on behalf of the Authors (23 Jun 2015)

Short summary

A sensitivity analysis is used to examine how error characteristics (type, distributions, and magnitudes) in meteorological forcing data impact outputs from a physics-based snow model in four climates. Bias and error magnitudes were key factors in model sensitivity and precipitation bias often dominated. However, the relative importance of forcings depended somewhat on the selected model output. Forcing uncertainty was comparable to model structural uncertainty as found in other studies.