the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Teaching hydrological modelling: Illustrating model structure uncertainty with a ready-to-use teaching module
- 1University of Saskatchewan Coldwater Laboratory, Canmore, Alberta, Canada
- 2Institute of Hydrology and Meteorology, Technische Universität Dresden, Dresden, Germany
- 1University of Saskatchewan Coldwater Laboratory, Canmore, Alberta, Canada
- 2Institute of Hydrology and Meteorology, Technische Universität Dresden, Dresden, Germany
Abstract. Estimating the impact of different sources of uncertainty along the modelling chain is an important skill graduates are expected to have. Broadly speaking, educators can cover uncertainty in hydrological modelling by differentiating uncertainty in data, model parameters and model structure. This provides students with insight on the impact of uncertainties on modelling results and thus on the usability of the acquired model simulations for decision making. A survey among teachers in the earth and environmental sciences showed that model structural uncertainty is the least represented uncertainty group in teaching. This paper introduces a teaching module that introduces students to the basics of model structure uncertainty through two ready-to-use exercises. The module is short and can easily be integrated into an existing hydrologic curriculum, limiting the time investment needed to teach this aspect of modelling uncertainty. A trial application at the Technische Universität Dresden (Germany) showed that the exercises can be completed in less than two afternoons and that the provided setup effectively transfers the intended insights about model structure uncertainty. The module requires either Matlab or Octave, and uses the open-source Modular Assessment of Rainfall-Runoff Models Toolbox (MARRMoT) and the open-source Catchment Attributes and Meteorology for Large-sample Studies (CAMELS) dataset.
- Preprint
(580 KB) -
Supplement
(842 KB) - BibTeX
- EndNote
Wouter J. M. Knoben and Diana Spieler
Status: closed
-
RC1: 'Comment on hess-2021-30', Anonymous Referee #1, 14 Feb 2021
In this contribution, Knoben and Spieler describe a teaching module on structural uncertainty in hydrological modelling. While I appreciate the attempt to provide this module to other hydrology teachers, I have several major concerns with the manuscript in its current status. I hope the authors can use my comments to redesign and improve their study and manuscript.
Major comments:
Selection of the two catchments in this study: I find the selection of the two catchments a bit problematic. Mainly, the two catchments vary in several aspects besides the so-called aridity fraction (e.g. size). This makes comparison difficult. Furthermore, one of the catchments reports zero-flows. Here it is important to note that the used model variants are by design not able to simulate zero-flows. The occurrence of zero-flows also makes the use of log-transformations for the computation of performance measures challenging.
I am also a bit confused by the selection of the two model variants, why just these two?
I am missing an evaluation of how successful the suggested module is. As it is now, basically the same claims that the authors make to motivate their module are also used to describe its success, which is not convincing. What would be needed is some form of evaluation by surveying students who took the class. I am also missing information on how many students and with which background participated in the course in Dresden.
The authors claim that their module could be added into ‘any hydrology course with minimal effort’ (P10L10). I'm afraid I have to disagree for several reasons:
- If at all, then it can be added to courses in hydrological modelling, but not all hydrology courses.
- If Matlab is not used in a particular class, including this module is by no means trivial
- Teaching materials are not provided; this would be important as a service to a potential teacher who wants to adapt this module in their course.
- The fixed selection of catchments and models might limit the utility of the module.
I would recommend describing the module first in generic terms. Both catchments and model variants could be left open to be selected as appropriate for a particular course. Forst of all, there is great value in using catchments that the students are familiar with. Using US catchments might not be the most pedagogical choice in many cases. Furthermore, depending on which programming language/modelling frameworks are used in a course, it might also be more useful to use an alternative to the option presented here. In a second step, a concrete implementation of the module could be described (=as it is described now) and guidance could be given on alternatives. Finally, it is crucial to evaluate the module in some way (e.g. student survey before-after)
Minor comments:
P3L26 (mathematically) accurate – I think you just mean ‘better’. Note that a model can be mathematically accurate but still totally useless.
P5L10 Aridity fraction: please explain this term and how it is computed
P8L2: The statement that instructions are straightforward is followed by a ‘fork and clone’ statement that might be not at all straightforward to most readers.
P10L4: formalize? Do you mean formulate?
P10L30: Does this mean it was an one-day course in practice?
P11L16: sorry, but the choice of one single student can’t be really used as a convincing argument
Figure 2 is hard to read and needs to be improved. I am olso a bit wondering about the shown precip data, for me it does not look as if “on average 294 days have < 1 mm precipitation” from this figure
- AC2: 'Reply on RC1', Wouter Knoben, 13 Mar 2021
-
RC2: 'Comment on hess-2021-30', Anonymous Referee #2, 22 Apr 2021
This manuscript explains a module prepared by the authors to teach students the concept of the model structural uncertainty. Along with description of the module, results of the survey designed by the author are discussed to show that the amount of time and effort put toward teaching this concept has been minimal among teachers in the earth and environmental sciences. I found the manuscript very well-written and easy to understand. Moreover, making the teaching module ready for other teachers is a big plus, making the work a potentially popular study among the community of hydrology teachers. However, the manuscript does not appear to fall under the scope of HESS that looks for studies that “contribute to the advancement of hydrological modelling, hydrological monitoring and data analysis, process concepts, experimental design and technology, or theoretical foundations”.
Moreover, I have the following two major comments can help authors improve their manuscript and making it more easily adaptable for other teachers.
- Students require clear directions on how to evaluate the models, but the manuscript does not discuss which directions should be given to students to evaluate the uncertainty. It is mentioned on page 8 line 25 that qualitative plots are used to visualize the results, but it is not clear what those plots are. Also, KGE was used as the calibration objective, but calibration is inherently a multi-objective optimization task. Therefore, I encourage the authors to discuss with more in-depth information about what directions should be given to students in this course to be able to evaluate the uncertainty.
- I believe the big missing pieces of puzzle in the module are:
- What should students do when they learn the fact that the model structural uncertainty exists? For example, should they discard all models but one? Or, should they select a sub-set of the models?
- How could students incorporate the estimated model structural uncertainty in their studies? For example, could they come up with a probabilistic estimation of the system response to hydrologic events?
- AC3: 'Reply on RC2', Wouter Knoben, 24 Apr 2021
Status: closed
-
RC1: 'Comment on hess-2021-30', Anonymous Referee #1, 14 Feb 2021
In this contribution, Knoben and Spieler describe a teaching module on structural uncertainty in hydrological modelling. While I appreciate the attempt to provide this module to other hydrology teachers, I have several major concerns with the manuscript in its current status. I hope the authors can use my comments to redesign and improve their study and manuscript.
Major comments:
Selection of the two catchments in this study: I find the selection of the two catchments a bit problematic. Mainly, the two catchments vary in several aspects besides the so-called aridity fraction (e.g. size). This makes comparison difficult. Furthermore, one of the catchments reports zero-flows. Here it is important to note that the used model variants are by design not able to simulate zero-flows. The occurrence of zero-flows also makes the use of log-transformations for the computation of performance measures challenging.
I am also a bit confused by the selection of the two model variants, why just these two?
I am missing an evaluation of how successful the suggested module is. As it is now, basically the same claims that the authors make to motivate their module are also used to describe its success, which is not convincing. What would be needed is some form of evaluation by surveying students who took the class. I am also missing information on how many students and with which background participated in the course in Dresden.
The authors claim that their module could be added into ‘any hydrology course with minimal effort’ (P10L10). I'm afraid I have to disagree for several reasons:
- If at all, then it can be added to courses in hydrological modelling, but not all hydrology courses.
- If Matlab is not used in a particular class, including this module is by no means trivial
- Teaching materials are not provided; this would be important as a service to a potential teacher who wants to adapt this module in their course.
- The fixed selection of catchments and models might limit the utility of the module.
I would recommend describing the module first in generic terms. Both catchments and model variants could be left open to be selected as appropriate for a particular course. Forst of all, there is great value in using catchments that the students are familiar with. Using US catchments might not be the most pedagogical choice in many cases. Furthermore, depending on which programming language/modelling frameworks are used in a course, it might also be more useful to use an alternative to the option presented here. In a second step, a concrete implementation of the module could be described (=as it is described now) and guidance could be given on alternatives. Finally, it is crucial to evaluate the module in some way (e.g. student survey before-after)
Minor comments:
P3L26 (mathematically) accurate – I think you just mean ‘better’. Note that a model can be mathematically accurate but still totally useless.
P5L10 Aridity fraction: please explain this term and how it is computed
P8L2: The statement that instructions are straightforward is followed by a ‘fork and clone’ statement that might be not at all straightforward to most readers.
P10L4: formalize? Do you mean formulate?
P10L30: Does this mean it was an one-day course in practice?
P11L16: sorry, but the choice of one single student can’t be really used as a convincing argument
Figure 2 is hard to read and needs to be improved. I am olso a bit wondering about the shown precip data, for me it does not look as if “on average 294 days have < 1 mm precipitation” from this figure
- AC2: 'Reply on RC1', Wouter Knoben, 13 Mar 2021
-
RC2: 'Comment on hess-2021-30', Anonymous Referee #2, 22 Apr 2021
This manuscript explains a module prepared by the authors to teach students the concept of the model structural uncertainty. Along with description of the module, results of the survey designed by the author are discussed to show that the amount of time and effort put toward teaching this concept has been minimal among teachers in the earth and environmental sciences. I found the manuscript very well-written and easy to understand. Moreover, making the teaching module ready for other teachers is a big plus, making the work a potentially popular study among the community of hydrology teachers. However, the manuscript does not appear to fall under the scope of HESS that looks for studies that “contribute to the advancement of hydrological modelling, hydrological monitoring and data analysis, process concepts, experimental design and technology, or theoretical foundations”.
Moreover, I have the following two major comments can help authors improve their manuscript and making it more easily adaptable for other teachers.
- Students require clear directions on how to evaluate the models, but the manuscript does not discuss which directions should be given to students to evaluate the uncertainty. It is mentioned on page 8 line 25 that qualitative plots are used to visualize the results, but it is not clear what those plots are. Also, KGE was used as the calibration objective, but calibration is inherently a multi-objective optimization task. Therefore, I encourage the authors to discuss with more in-depth information about what directions should be given to students in this course to be able to evaluate the uncertainty.
- I believe the big missing pieces of puzzle in the module are:
- What should students do when they learn the fact that the model structural uncertainty exists? For example, should they discard all models but one? Or, should they select a sub-set of the models?
- How could students incorporate the estimated model structural uncertainty in their studies? For example, could they come up with a probabilistic estimation of the system response to hydrologic events?
- AC3: 'Reply on RC2', Wouter Knoben, 24 Apr 2021
Wouter J. M. Knoben and Diana Spieler
Wouter J. M. Knoben and Diana Spieler
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 1,045 | 546 | 21 | 1,612 | 113 | 8 | 10 |
- HTML: 1,045
- PDF: 546
- XML: 21
- Total: 1,612
- Supplement: 113
- BibTeX: 8
- EndNote: 10
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1