Using expert knowledge to increase realism in environmental system models can dramatically reduce the need for calibration

Gharari, S.; Hrachowitz, M.; Fenicia, F.; Gao, H.; Savenije, H. H. G.

doi:https://doi.org/10.5194/hess-18-4839-2014

Articles | Volume 18, issue 12

https://doi.org/10.5194/hess-18-4839-2014

© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/hess-18-4839-2014

© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 18, issue 12

Research article

|

05 Dec 2014

Research article |

| 05 Dec 2014

Using expert knowledge to increase realism in environmental system models can dramatically reduce the need for calibration

S. Gharari, M. Hrachowitz, F. Fenicia, H. Gao, and H. H. G. Savenije

Download

Final revised paper (published on 05 Dec 2014)
Supplement to the final revised paper
Preprint (discussion started on 05 Dec 2013)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

SC C7361: 'comments to hess-2013-519 research article', Wei Gong, 22 Jan 2014
- AC C8076: 'Reply to short comments by Dr. Gong', Shervan Gharari, 03 Mar 2014
RC C7535: 'Comments on hess-2013-519', Anonymous Referee #1, 29 Jan 2014
- AC C8080: 'reply to comments by anonymous reviewer 1', Shervan Gharari, 03 Mar 2014
RC C7571: 'Review comments', Anonymous Referee #2, 31 Jan 2014
- AC C8082: 'reply to comments by anonymous reviewer 2', Shervan Gharari, 03 Mar 2014
AC C8092: 'Letter to the editor', Shervan Gharari, 03 Mar 2014

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Publish subject to minor revisions (Editor review) (22 Mar 2014) by Fuqiang Tian

Generally I like the manuscript very much together with the simultaneously submitted technical note. They are clear and well-written as commented by the reviewers. I am also very happy to see the insightful discussions on the papers. I recommend its publication after properly and positively addressing all the comments raised by all the reviewers.
The major issue is whether this manuscript and the accompanying submitted technical note should be merged or published separately. According to the reviewers' comments and the careful review on my own, I suggest the authors combine the two manuscripts. The reasons are (maybe not include all the reviewer's comments):
1) It seems to me that the expert knowledge the authors used in this manuscript includes the following two aspects: the three function units based on topography that can represent the hydrological function better for a catchment, and the constraints (parameter constraints and process constraints). The latter one is very much overlapped with the technical note. For the first aspect, to my understanding the expert knowledge is always explored to improve the model structure (explicitly and implicitly), i.e., the realism in environmental system models. The incentive of many model development is inspired by some new expert knowledge, like the topography index (HAND) used in FLEX-TOPO model. The knowledge of functional unit division based on topography is discussed in the previous paper and not the original contribution in this manuscript. The constraint knowledge by itself is not novel enough for publication, especially separately from the technical note.
2) This manuscript itself cannot demonstrate the actual value of the proposed approach (see comment from second anonymous reviewer). I agreed with reviewer #2 that the test is not enough by a single catchment, a single calibration/validation, and a very short period. The readers would not be convinced by such simple test of a new approach. Also, the authors submitted separate manuscripts to HESS and WRR on the expansion work to a large basin. It is not appropriate to borrow more details into this manuscript. I would suggest the authors combine the two manuscripts together and focus on the constraint approach.
Thanks.

Hide

AR by Shervan Gharari on behalf of the Authors (25 Jun 2014) Author's response Manuscript

ED: Reconsider after major revisions (04 Jul 2014) by Fuqiang Tian

AR by Shervan Gharari on behalf of the Authors (05 Jul 2014) Author's response Manuscript

ED: Referee Nomination & Report Request started (13 Jul 2014) by Fuqiang Tian

RR by Anonymous Referee #1 (15 Jul 2014)

RR by Anonymous Referee #2 (19 Aug 2014)

Suggestions for revision or reasons for rejection

The revised version of the manuscript is clearer. The main change the authors made was to extend the test period from 2 to 4 years. I found interesting the main conclusion of the manuscript that the more complex constrained but uncalibrated model structure can provide as good results as the calibrated lumped version.

That said, I felt quite disappointed by the way my comments (Reviewer #2) were accounted for by the authors. Several comments were judged “out of the scope” of the article although I still think they are not. At least a few lines of discussion could have been added on these issues (e.g. previous comments #6, 9, 15). Others (e.g. comments #3, 9) were said to be already discussed in the two recent papers published by the authors (Gao et al., 2013 and Hrachowitz et al., 2014) but this is still not mentioned nor discussed in the revised article (though I noticed these two papers are now cited in the companion paper). For a few other minor comments, the authors said they agree but I did not see the corresponding changes in the revised text (e.g. #1, 11). Last, it seems that some parts of the discussion of results in the revised manuscript were not updated following the introduction of the new results, with comments that are now not supported by results shown (see detailed comments).

Overall, my main concern expressed about the robustness of the conclusions still remains. Although the proposed approach seems sensible, the authors could have done a bit more to make their article not “just a simple test” as mentioned in their reply.

Since the authors do not seem willing to deeply modify their manuscript, I just advise the few minor revisions below.

Detailed comments
1. Page 1, Line 48: “are provided”

2. Page 1, Line 48: “Grayson”

3. Page 3, Line 34: “and thereby limit predictive uncertainty” (?)

4. Page 4, Line 28: “Drogue”

5. Page 7, Line 8: “reservoirs” (?)

6. Page 15, Line 36: "constraints"

7. Page 17, Lines 22-27: Although this comment was true in the first version of the manuscript, this is unclear now with the new results. The performance of the FLEXA model only degrades on the ENS criterion. Performance even improves for ENS,log and ENS,FDC between calibration and validation for the three models. Of course, this may happen when the validation period is easier to model than the calibration one. But then the notion of “validation/calibration performance ratio” becomes difficult to interpret. The interpretation would have been easier if the authors had chosen to apply the full split sample test as already suggested in my previous review comments, since it would ease the comparison of performance between calibration and validation (even though I agree with the authors reply that in the case of uncalibrated models, calibration and validation performance would be the same). Note that the slight change the authors made in their test period changed the way the results should be interpreted, which reinforce my previous feeling that conclusions given in the article could be more robust with enhanced testing scheme.

8. Page 17, Lines 28-33: Again, this statement is less clear now with the new results (FLEXB is better than FLEXC for two criteria) and this conclusion should be revised.

9. Page 19, Lines 8-9: According to results shown in Table 4, this statement is false since the median calibration performance is not better for FLEXA than FLEXB and the uncertainty is the largest for FLEXA.

10. Page 19, Lines 13-15: This statement is not supported by the results shown since FLEXB appears better performing than FLEXC.

11. Figure 5: The caption could indicate which distribution percentiles are shown by the box-plot.

12. Page 20, line 17: “reaction tends to”

13. Figure 6: Should this figure better show the results in validation, which are more representative of actual model performance than calibration results?

Hide

RR by Anonymous Referee #3 (25 Aug 2014)

Suggestions for revision or reasons for rejection

This article is very interesting and well-written. The authors provide a systematic and potentially useful way to incorporate the expert or soft knowledge in parameter identification. I advise publication after minor revision.
Specific comments
1.Section 4-1, the model FLEXC having the best performance with the un-calibrated constrained parameters does not necessarily prove that “the imposed relational constraints force the model and its parameters towards a more realistic behavior”. The author should do the comparison among these different models with calibrated parameters (using conventional calibration methods, avoiding the effects of these proposed constraints). In my opinion, it is the additional landscape or HRU that improves the simulation performance, far more than the additional constraints.
2.Section 4-3, the second paragraph, “the expectation that increasingly complex models will have increasingly poor validation/calibration performance ratios”, many similar expectation results have been mentioned in this article, so it is necessary to provide at least one figure to show these expectation results are reasonable. For example, compare the validation results and uncertainty between these three models with calibrated parameters using a conventional method.
3.Section 4-3, the third paragraph, in Figure 5 (Ens), the difference between a calibrated lumped model FLEXA and a more complex constrained but uncalibrated model FLEXC in validation period is small, but when we compare table3 and table 4, the difference between the results of Ens(log) is large (0.75 vs 0.63), and the difference of uncertainty is also large. So the following implication is not reasonable. Please add more analysis and discussion based on the other two metrics.
4.Although this article focuses on the constraint-based method, I want to know how big is the gap between the constraint-based method and the conventional calibration method (a benchmark), and thus can estimate the potential capacity of the proposed method.

Hide

ED: Publish subject to minor revisions (Editor review) (30 Aug 2014) by Fuqiang Tian

AR by Shervan Gharari on behalf of the Authors (27 Oct 2014) Author's response Manuscript

ED: Publish as is (07 Nov 2014) by Fuqiang Tian

AR by Shervan Gharari on behalf of the Authors (12 Nov 2014) Manuscript