On the assimilation of environmental tracer observations for model-based decision support

Knowling, Matthew J.; White, Jeremy T.; Moore, Catherine R.; Rakowski, Pawel; Hayley, Kevin

doi:https://doi.org/10.5194/hess-24-1677-2020

Articles | Volume 24, issue 4

https://doi.org/10.5194/hess-24-1677-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-24-1677-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 24, issue 4

Research article

|

08 Apr 2020

Research article |

| 08 Apr 2020

On the assimilation of environmental tracer observations for model-based decision support

Matthew J. Knowling, Jeremy T. White, Catherine R. Moore, Pawel Rakowski, and Kevin Hayley

Download

Final revised paper (published on 08 Apr 2020)
Supplement to the final revised paper
Preprint (discussion started on 18 Sep 2019)
Supplement to the preprint

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

SC1: 'Excellent line of investigation/application', Ty P. A. Ferre, 20 Sep 2019
- SC2: 'Reply to Prof. Ty Ferre', Matthew Knowling, 22 Sep 2019
RC1: 'A more formal review', Ty P. A. Ferre, 04 Oct 2019
- SC5: 'Reply to Prof. Ty Ferre's Referee Comment', Matthew Knowling, 19 Oct 2019
- AC1: 'Consolidated Replies to Reviewer Comments', Matthew Knowling, 02 Dec 2019
SC3: 'Comments on paper HESS-2019-436', Chris Turnadge, 11 Oct 2019
- SC4: 'Reply to Chris Turnadge', Matthew Knowling, 15 Oct 2019
RC2: 'Review for "On the assimilation of environmental tracer observations for model-based decision support" by Knowling et al.', Anonymous Referee #2, 13 Nov 2019
- SC6: 'Reply to Anonymous Referee #2', Matthew Knowling, 18 Nov 2019
- AC1: 'Consolidated Replies to Reviewer Comments', Matthew Knowling, 02 Dec 2019

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (further review by editor and referees) (13 Dec 2019) by Fabrizio Fenicia

AR by Matthew Knowling on behalf of the Authors (15 Dec 2019) Author's response Manuscript

ED: Publish subject to revisions (further review by editor and referees) (18 Jan 2020) by Fabrizio Fenicia

AR by Matthew Knowling on behalf of the Authors (18 Jan 2020) Author's response Manuscript

ED: Publish subject to revisions (further review by editor and referees) (21 Jan 2020) by Fabrizio Fenicia

ED: Referee Nomination & Report Request started (24 Jan 2020) by Fabrizio Fenicia

RR by Anonymous Referee #2 (21 Feb 2020)

Suggestions for revision or reasons for rejection

This is a review of the revised version of "On the assimilation of environmental tracer observations for model-based decision support" by Knowling et al.

I have carefully read the revised manuscript and the authors' responses and believe that the revised manuscript will make a valuable contribution to HESS, provided that minor points in the presentation of the results and discussion of existing literature are considered.

My main concerns with the original manuscript were a.) a lack of information and/or unclear descriptions of the modelling/history matching/uncertainty quantification procedures applied in case study 1, b.) a lack of information on observations used for case study 2, and c.) that the relevance of the findings were overstated.

The authors have done a good job in addressing concerns a.) and b.) through appropriate revisions, additions and clarifications. With respect to point c.), the manuscript has improved but not sufficienly. The context of the findings is now much clearer than in the original manuscript, owing to the improved description of observation data, history matching procedures, and uncertainty quantification approaches. I find the findings of this study interesting and relevant, i.e., that a.) tritium-based MRT are not adding a tremendous amount of information to an already rich dataset consisting of hydraulic heads and spring discharge observations for a model aimed at predicting spring discharge (case study 1), and that b.) tritium concentration observations can lead to parameter overcompensation and thus forecasting bias when used to calibrate a two-layer model aimed at predicting nitrate load, where in reality the complex hydrogeological situation demands for a much more complex model structure. However, these findings are presented in an overstating manner. On the one hand side, the authors acknowledge in their introduction that it is already widely understood that the value of a specific observation type is highly dependent on the system modeled, the conceptual and numerical model employed, and the forecast being made (among other controlling variables). Multiple studies and review articles which have already made this point are discussed in the introduction. However, on the other hand side, despite acknowledging this previous work, the authors at the same time present their own findings as if these previous studies and reviews do not exist, suggesting that all others unqualifiedly advocate for more use of environmental tracers without knowledge/critical discussion of the potential pitfalls associated with the use of environmental tracers for environmental model calibration. However, this is completely wrong, as the previous studies and reviews all DO discuss various the potential pitfalls associated with many different observation types. The following sentence from the abstract (lines 17-18) is an example of the overstatement of the findings of this study and the obscuring of findings presented in existing literature:

'The findings of this study challenge the unqualified advocacy of the increasing use of tracers, and diverse data types more generally, whenever environmental model data assimilation is undertaken with imperfect models'.

A large number of studies and review articles explored the worth of diverse data types for environmental model calibration in a scientifically sound manner and justifiably demonstrated that diverse data in many contexts contain highly valuable information for the reduction of parameter and forecast uncertainty of environmental models. This sentence, and others alike throughout the manuscript, are inappropriate and oversell the findings of the present study. Another example is the following sentence from the introduction (lines 48-49):

'However, the notion that unabated assimilation of diverse data types (including environmental tracers) is always of benefit holds only from a theoretical standpoint'.

This sentence is surprising, because it directly follows the discussion of previous studies and reviews that investigate and highlighted pitfalls of using tracer-based observations in environmental model calibration. The apparent 'notion that unabated assimilation of diverse data is always of benefit' does not exist in the referenced literature. Another statement of the same kind is found in the discussion and conclusions section:

Lines 311-312: 'We consider this recommendation to be in stark contrast to the common belief that “calibrating to more data improves the model and its predictions”, and therefore of significant implication to decision-support environmental modeling practitioners.'

The many existing studies and reviews on the topic are evidence that there already is a widespread understanding that the worth of a given observation in a given context is observation-, context-, model- and forecast- specific. In the most recent review on the matter, Schilling et al. (2019) highlight specifically with respect to (mean) residence time (RT) and travel time (TT) observations: 'The above findings confirmed that observations of TT and RT can be beneficial for flow model calibration but that their implementation into a flow model calibration routine requires a significant number of assumptions to be made and, therefore, is associated with uncertainties.'

The sentence on lines 292-293 is therefore clearly wrong: 'These findings are nevertheless highly relevant in that MRT observations are widely regarded to be of benefit in constraining uncertain model parameters more generally (Schilling et al., 2019)—regardless of the forecast.'

In contrast to this misleading statement on lines 292-293, Schilling et al. (2019) further noted: 'Also, observations of RTs and TTs contain valuable information about SW‐GW flow systems, but their implementation into flow model calibration needs to be done with great care, as it is associated with many potential pitfalls. Unlike exchange flux observations, which do not rely on many underlying assumptions and do not require additional processes to be simulated, the successful implementation of observations of RT and TT into flow model calibration is typically based on many underlying assumptions on the conceptual model and on simplified representations in flow models.' Further: 'However, one should exercise due care when choosing to apply a transformation or a simplified process representation in flow models: Depending on the spatial and temporal scale and the nature of the original observation, transformations and simplifications might result in an over-simplification, and calibration might result in a biased model, of which the structural defects cannot be quantified (see Turnadge & Smerdon, 2014). Transformed observations should thus only be considered for cases where direct integration of the untransformed observations is impossible or unfeasible, or where the transformation is not associated with substantial additional uncertainty.'

The existing literature on the matter, here represented by the most recent review by Schilling et al. (2019), clearly discuss the pitfalls associated of using diverse and tracer-based data, and particularly of transformed observations such as (mean) residence times. The statements about the current 'notion', 'common belief' and 'unqualified advocacy' made in this manuscript are thus evidently obscuring the findings of existing literature on the matter and overselling the findings of the present study. This is not only unnecessary, it is inacceptable conduct.

As elaborated above, the findings of the present study are interesting and relevant and will make a valuable contribution to HESS, but the manuscript must not make misleading statements about existing literature in an attempt to overstate the relevance of the study's findings. All the misleading statements highlighted herein must be removed before this article can be considered for final publication. Minor revisions are therefore necessary prior to publication.

Hide

ED: Publish subject to minor revisions (review by editor) (28 Feb 2020) by Fabrizio Fenicia

AR by Matthew Knowling on behalf of the Authors (29 Feb 2020) Author's response Manuscript

ED: Publish as is (12 Mar 2020) by Fabrizio Fenicia

AR by Matthew Knowling on behalf of the Authors (12 Mar 2020) Author's response Manuscript

Short summary

The incorporation of novel and diverse data sources into predictive models is expected to improve the reliability of model forecasts. This study critically and rigorously explores the extent to which this expectation holds given the imperfect nature of numerical models (and therefore their compromised ability to appropriately assimilate information-rich data). We show that environmental tracer observations may be of variable benefit in reducing forecast uncertainty and may induce forecast bias.