airGRteaching: an open-source tool for teaching hydrological modeling with R
Hydrological modeling is at the core of most studies related to water, especially for anticipating disasters, managing water resources, and planning adaptation strategies. Consequently, teaching hydrological modeling is an important, but difficult, matter. Teaching hydrological modeling requires appropriate software and teaching material (exercises, projects); however, although many hydrological modeling tools exist today, only a few are adapted to teaching purposes. In this article, we present the
airGRteaching package, which is an open-source R package. The hydrological models that can be used in
airGRteaching are the GR rainfall-runoff models, i.e., lumped processed-based models, allowing streamflows to be simulated, including the GR4J model. In this package, thanks to a graphical user interface and a limited number of functions, numerous hydrological modeling exercises representing a wide range of hydrological applications are proposed. To ease its use by students and teachers, the package contains several vignettes describing complete projects that can be proposed to investigate various topics such as streamflow reconstruction, hydrological forecasting, and assessment of climate change impact.
In order to anticipate and manage water conditions, outcomes of hydrological research are applied on a regular basis by water managers and stakeholders. These are aimed at addressing numerous challenges, such as the following:
The consequences and damage of extreme events (floods and droughts) are more limited when such events are better anticipated or managed. Hydrological science can also help to optimize profits in the hydropower sector (Cassagnole et al., 2021). In this context, hydrological models are key tools because they help to transform meteorological variables into hydrological variables.
1.1 On the need for (and relevance of) teaching hydrology using models
For many years, teaching hydrology has implied teaching hydrological modeling (Wagener and McIntyre, 2007). As a consequence, teaching hydrology can also imply programming, thereby raising the important issues of automatic calibration, sensitivity analysis (AghaKouchak and Habib, 2010; Knoben and Spieler, 2022), and also reproducibility in hydrology (Hutton et al., 2016). Given the advantages of applying hydrological models for the real-life cases listed above, there is a considerable interest in and need for models to teach hydrology. First, hydrological modeling is a daily task for numerous practitioners, and thus it is an art that needs to be understood and mastered by students. Moreover, models are key tools for understanding the hydrological cycle, the interactions between the processes involved, and how hydrological variables evolve. Lastly, models represent an efficient way of proposing “active learning” courses to students. Thus, the impact of using hydrological models with students while they are learning can be significant. Sanchez et al. (2016) showed that the use of a simple spreadsheet with real hydrological data had a significant and positive impact on the civil engineering curriculum. AghaKouchak and Habib (2010) also found significant learning gains for students using modeling tools in class. Nevertheless, the added value of using models in class is not automatic and straightforward. For example, Marshall et al. (2015) demonstrated that the same hydrological course offered using either (i) Microsoft Excel (Microsoft Corporation, 2019), (ii) MATLAB (2018), or (iii) the COMSOL Multiphysics software (Zimmerman, 2006) (https://www.comsol.com/, last access: 30 December 2022) made no significant difference in student performances. This result highlights the need to use tools tailored for teaching hydrology with models.
1.2 On the need for common tools for teaching hydrological (reproducible) modeling
Wagener and McIntyre (2007) and Merwade and Ruddell (2012) highlighted the large diversity of approaches available to teach hydrology. Hutton et al. (2016) argued for the need for reproducible computational hydrology, to teach version-controlled programming:
A key step to change this culture is to ensure that computational science training (e.g., http://software-carpentry.org) is properly embedded within hydrological science curriculums, so that future generations of hydrologists have the skills to build readable, version controlled and unit-tested software (McConnell, 2004), allowing them to engage more fully in an open scientific community by reproducing and reusing each other’s research outputs.
This moves toward reproducible hydrology (Hall et al., 2022) and leads to the emergence of experiments of virtual laboratories (Ceola et al., 2015; Tarboton et al., 2014), open-source software (Coron et al., 2017; Slater et al., 2019), and open datasets (Addor et al., 2017; Irving et al., 2018). What about open hydrological teaching?
1.3 A review of modeling tools designed for teaching hydrological modeling
The development of modeling tools dedicated to teaching hydrology began in the 1960s, with the pedagogic hydrological model
ABC (Fiering, 1967; Kay et al., 1982; Burt and Butcher, 1986; Kirkby and Naden, 1988). Since the development of
ABC, several software programs have been designed for teaching hydrology (see HESS Special Issue entitled “Hydrology education in a changing world”, Seibert et al., 2013). Elshorbagy (2005) used a system dynamics approach based on the
STELLA visual programming language (Richmond et al., 1985) for teaching watershed hydrology. Pérez-Sánchez et al. (2022) described the use of Microsoft Excel (Microsoft Corporation, 2019) spreadsheets for teaching hydrological modeling and for estimating climate change impacts in a postgraduate civil engineering master's degree. The
HBV rainfall-runoff model has been used several times as a basis to develop an education-dedicated version: AghaKouchak and Habib (2010) and AghaKouchak et al. (2013) developed the
HBV-EDU toolbox in MATLAB to teach hydrology and uncertainty estimation (https://fr.mathworks.com/matlabcentral/fileexchange/41395-hbv-edu-hydrologic-model?s_tid=FX_rc1_behav, last access: 30 December 2022), while Seibert and Vis (2012) created the
HBV-light software. Mendez and Calvo-Valverde (2016) and Viglione and Parajka (2020) developed, respectively,
TUWmodel within the R programming language (R Core Team, 2023), and several web applications designed for using HBV are available online (e.g., https://github.com/NikoZHAI/lumphydro, last access: 30 December 2022). This approach of simplifying an existing hydrological model for teaching purposes has been applied with
HBV but also with other models such as
VIC by Wi et al. (2017) with
VIC-ASSIST (developed in MATLAB). The MATLAB-based
HMETS model (Martel et al., 2017) (https://fr.mathworks.com/matlabcentral/fileexchange/48069-hmets-hydrological-model?s_tid=FX_rc1_behav, last access: 30 December 2022), initially developed for teaching, has proved to be efficient over a large sample of 320 catchments located in the contiguous United States.
Numerous solutions exist to teach hydrological modeling, but they all have their limitations (see Carriba Demange et al., 2022), such as being a “light version” of a model (e.g.,
HBV-light), having an inability to import one's own data (e.g.,
TUWteaching, https://webaapptuwmodel.shinyapps.io/TUWteaching/, last access: 30 December 2022), having an inability to access and modify the source code (e.g.,
RS MINERVE; García Hernández et al., 2020), having an inability to manually or automatically calibrate the model parameters (e.g.,
HBV.IANIGLA; Toum et al., 2021), or being based on proprietary programming language (e.g.,
VIC-ASSIST developed in MATLAB).
1.4 R, a language increasingly used by hydrologists, especially for modeling …
The open-source programming language R is one of the most widely used languages in the hydrological community. It offers many open-source libraries useful, for example, for retrieving hydro-meteorological data, performing spatial analysis, and analyzing hydrological statistics. The whole workflow undertaken in hydrological studies can be done with R (see Slater et al., 2019), which is very useful for practical reasons. The reader is asked to refer to Slater et al. (2019) for further details about the advantages of R for all the steps of the workflow and to the R Hydrology Task View (https://cran.r-project.org/web/views/Hydrology.html, last access: 1 August 2022, Zipper et al., 2022) for a complete list of R packages linked to hydrology. The choice of hydrological modeling R packages is particularly large (see Astagneau et al., 2021, for a recent review), providing a variety of solutions adapted to the diverse problems or case studies that can be encountered. Here again, the reader is referred to Astagneau et al. (2021) for further details about the packages and models available. In addition, R facilitates interdisciplinary work in the other fields of geosciences in which R is also used (e.g., Bezak et al., 2019, who use the
airGR package, Coron et al., 2017, 2022, for hydrological modeling and the prediction of landslides). One of the strengths of R is its ability to incorporate geographic data and spatial analysis, such as in the use of the MODIS dataset, for example, for modeling of snow accumulation and melt (Riboust et al., 2019).
1.5 … but not yet for teaching, even if attempts are being made
A basic search with the keywords “educ*” and “teach*” (last check on 1 August 2022) in the R Hydrology Task View only returns a few packages that address teaching aspects of hydrology:
TUWmodel (Viglione and Parajka, 2020), which contains a hydrological model that is proposed for educational purposes but does not contain actual exercises or an interface;
EcoHydRology (Fuka et al., 2018), which is aimed at providing a flexible framework for hydrology-related staff, students, or researchers for basic exercises; and
airGRteaching (Delaigue et al., 2018, 2023b), which is the topic of the present article.
airGRteaching relies on the widely used GR hydrological models, a family of rainfall-runoff models simulating streamflows that are usually used in lumped mode (i.e., running at the basin scale with aggregated input) and that were recently incorporated into an R package (
airGR; Coron et al., 2017, 2022). To provide teaching material, the
airGR developers set up an add-on package dedicated to teaching hydrology, named “
airGRteaching”. This package contains a graphical user interface, simple functions, and hydrology exercises. Since then it has been used for teaching and for hands-on projects in various universities and engineering schools (see, for instance, a master's degree project using
airGRteaching: Roux and Brigode, 2018).
airGRteaching relies on the widely used GR models (see, e.g., Perrin et al., 2003, which presents the widely known GR4J model) and on the
airGR package, which has gained lots of interest over the past few years (see Coron et al., 2017, which presents the
airGR package or the list of publications on the airGR website (https://hydrogr.github.io/airGR/page_publications.html#Use_and_mention_of_airGR, last access: 30 December 2022) that lists all known uses of or references to
airGR), we believe that this tool can pave the way to developing new hydrology teaching initiatives, developing similar tools, and promoting hydrology to a more general audience.
In this paper, after introducing the general concepts taught in hydrology, we present the main features of the
airGRteaching package and introduce several exercises using this package.
2.1 The rationale behind
airGRteaching: a glance backward
The GR models were initially developed in the 1980s by Claude Michel and his colleagues at Cemagref (that recently became Irstea and then INRAE). The main objective was to design efficient models, starting from a simple structure and gradually adding complexity that proved useful for improving the model's predictive power (Michel, 1983). This approach prioritized predictive power over explanatory models (Shmueli, 2010), finding justification for this from results obtained using large data sets and not from predefined concepts. This led to the development of a family of models that are usually used in lumped mode (i.e., running at the basin scale with aggregated input).
To disseminate their models beyond the Fortran programming community, a long time ago, the developers of the GR models proposed Microsoft Excel spreadsheets containing hydrological models, namely, the GR1A, GR2M, and GR4J models, as well as the CemaNeige snow accumulation and melt model (see next section for a description of these models), accompanied by a dummy dataset (https://gitlab.irstea.fr/HYCAR-Hydro/ExcelGR, last access: 20 July 2023). The rationale behind this approach was dual: easily providing the GR models to external users (researchers and consultants from France and abroad) and illustrating the hydrological concepts to students with the models developed in-house. The relatively high efficiency and low computational time requirements of these models made them easy to run with Microsoft Excel. In addition, the use of Excel macros enabled interactivity (e.g., the possibility to automatically update simulations when parameter values are modified by users), and graphs were predefined.
airGR R package was developed to propose additional GR models, and the
airGRteaching R package was built as an add-on package of
airGR. These tools are described in the next sections.
2.2 The GR models and the
To ease the implementation of the GR models, Coron et al. (2017, 2022) proposed the
airGR package. Gathering seven hydrological models and one snow accumulation and melt model,
airGR can be seen as a research tool, as an efficient way for its developers to share research results, and as a tool simple enough to be used by water managers. The hydrological models included in
airGR differ in their complexity and time step, with a gradual increase in complexity as the time step decreases, and various application objectives:
GR1A (Mouelhi, 2003; Mouelhi et al., 2006a) is an annual one-parameter model, used for water resources assessment (Baahmed et al., 2015; Kouassi et al., 2012). It consists of a single equation relating the annual streamflow to antecedent annual precipitation and potential evapotranspiration.
GR2M (Mouelhi, 2003; Mouelhi et al., 2006b) is a monthly two-parameter model, used for water resources assessment and water regime modeling (Belarbi et al., 2017; Marchane et al., 2017). It consists of two stores: a production store used for calculating the part of rainfall transformed into discharge (effective rainfall) and a routing store used for distributing in time the effective rainfall toward the catchment outlet.
GR4J (Perrin et al., 2003) is a daily four-parameter model, used for water resources assessment, flood and drought simulation, and forecasting and climate change impact (Chauveau et al., 2013). In addition to the GR2M components, it contains two unit hydrographs that refine the temporal distribution of effective rainfall.
GR5J (Le Moine, 2008) is a daily five-parameter model, used for similar applications as GR4J. Compared to GR4J, GR5J contains only one unit hydrograph, and the intercatchment groundwater exchange function is slightly more general with two-way exchange fluxes between surface and regional groundwater.
GR6J (Pushpalatha et al., 2011) is a daily six-parameter model, used for similar applications to GR4J and GR5J. Compared to GR5J, an additional exponential store improves the representation of low flows.
CemaNeige (Valéry et al., 2014) is a daily two-parameter snow accumulation and melt model, used for snowy catchments. It consists of (i) a partition of precipitation into rainfall and snowfall upgraded with an extrapolation based on altitudinal gradients, (ii) a snow store that also represents the snow heat content, and (iii) a melt function. Optionally, satellite snow data can be used to calibrate an improved version of CemaNeige representing the snow water equivalent–snow cover area hysteresis relationship (Riboust et al., 2019).
Semi-distribution is enabled for the aforementioned models (except GR1A), which are originally lumped, in order to represent spatially heterogeneous catchments. The streamflow simulated for upstream catchments is propagated downstream using a lag function (de Lavenne et al., 2019).
airGRteaching embeds the main features of
airGR and offers simplified ergonomics. It therefore uses its basic tools, meaning that all models implemented in
airGR are available in
airGRteaching. Since these models have relatively simple structures and few parameters, they can be more easily understood by novice users such as students.
airGRteaching does not provide “simplified” versions of existing GR models. Thus, students are able to learn hydrological modeling from the same models that are used in practice, not from degraded versions.
To ease hands-on experience, the choice was made to reduce the number of functions to implement a complete modeling exercise (an
airGRteaching function therefore embeds several
airGR functions). In addition, the number of modeling options has been reduced, which limits the number of arguments to be specified for running a simulation and simplifies the associated documentation. All these choices allow users to focus on the main questions that beginners ask themselves when they start dealing with hydrological modeling.
airGRteaching contains only a few functions, which can be split into two groups:
a small set of functions to prepare data, to calibrate and run hydrological models, and to plot outputs, i.e., the basic functions needed to undertake a hydrological modeling study;
a function to launch a graphical user interface (GUI) to set up the hydrological models manually.
These two levels of use allow teachers to choose between different levels of technical difficulty. They can choose the most adapted use according to the time available for the exercises, the teaching objectives, and the students' skills.
To get started with the package, particular attention was given to the documentation. The user manual describes the implementation of functions precisely and succinctly and provides simple examples (https://cran.r-project.org/web/packages/airGRteaching/airGRteaching.pdf, last access: 30 December 2022). In addition, a website was created to explain how to use the different features step by step and to answer frequently asked questions (https://hydrogr.github.io/airGRteaching/, last access: 30 December 2022).
Table 1 summarizes the
2.4.1 Basic functions for undertaking a hydrological modeling study
The main steps required to undertake a hydrological modeling study can be performed with
airGRteaching with the help of a few simple functions:
A data preparation function,
PrepGR(). With only three main arguments, namely, the hydrometeorological input data as a data frame or independent vector time series, the name of the rainfall-runoff model to run, and a Boolean indicating whether the
CemaNeigesnow model is activated, this function prepares all the necessary inputs in the correct format for the
airGRteachingfunctions. If CemaNeige is activated, additional arguments are needed (e.g., catchment elevation distribution).
A calibration function,
CalGR(). With three main arguments, namely, the object produced by
PrepGR(), the objective function name (i.e., which criterion is used to optimize the parameter values), and the calibration period start and end, this function calibrates the chosen GR model. If desired, a transformation of discharge can be chosen for the objective function calculation in order to give more weight to certain ranges of discharges (Santos et al., 2018), and a warm-up period can also be defined.
A simulation function,
SimGR(). With four main arguments, namely, the object produced by
PrepGR(), the parameter values (output of
CalGR()or defined by the user), the name of an efficiency criterion used to evaluate the simulation, and the simulation period start and end, this function runs the chosen GR model and assesses its performance. If desired, a transformation can be used for the criterion calculation, and a warm-up period can be defined.
Static and dynamic functions,
dyplot(). These functions take any of the objects produced by
SimGR()as main arguments (to be chosen). Graph-tuning arguments are available but optional. The dynamic graphs show the observed and simulated discharge time series. The static graphs render a choice of graphs to be selected with the
whichargument. Dynamic graphs use the functionalities of the
dygraphspackage (Vanderkam et al., 2018).
Many graphical outputs are available (see Appendix A and B). Figure A1 provides a general overview of the precipitation and streamflow records to identify possible outliers and periods with missing data. Figure A2 adds the simulated streamflow to the previous graph in order to have an overall view of the calibrated model behavior and provides graphical diagnostic tools to check whether the simulated streamflow hydrograph fits the observed streamflow hydrograph. Figure A4 focuses on time series graphs (available in Fig. A2) and adds the potential evapotranspiration. Figure A3 focuses on the errors of the model compared to the observed streamflows. Figure A5 helps the concept of parameter optimization to be understood by displaying the tested parameter values and the correspondence with the value of the criterion chosen as objective function. In general, dynamic graphs (Figs. B1 and B2) help the values of time series to be read more precisely and to zoom in on a particular event for each of the two axes (some options are available, e.g., to add a rolling average or a time range selector).
2.4.2 The graphical user interface
Using the functionalities of the
shiny package (Chang et al., 2022), the
airGRteaching graphical user interface (GUI) called with the
ShinyGR() function allows one to use the GR models with no programming skills at all, thanks to an intuitive interface. The
ShinyGR() function takes hydrometeorological data and the simulation period start and end as arguments. Additional arguments can be provided if snow is present. Data can be provided for several catchments, and the function offers the possibility to use different themes for the interface. The GR and CemaNeige models and the objective function are to be selected within the interface.
Figure 1 presents a commented example of the interface. Several intuitive elements can be found. On the left side are the following:
“Choose a dataset” enables a dataset to be selected from those provided by the user to the function.
“Choose a model” enables a model to be selected among the GR2M monthly model and the GR4J, GR5J, and GR6J daily models according to the time step of the datasets provided (models at other time steps are not included in the GUI) and to activate the CemaNeige snow accumulation and melt model for the daily models.
“Parameter values” enables the parameters of the models to be modified. The parameters proposed are automatically adapted to the chosen model and the ranges are predefined. Changing any parameter value causes a real-time update of the plots and displayed scores (see below).
“Automatic calibration” enables an automatic calibration to be performed by optimizing a chosen objective function (among the Nash–Sutcliffe efficiency (NSE) and the Kling–Gupta efficiency (KGE) and with a squared root, inverse, or no transformation of discharge).
The following options are at the top:
“Choose a plot” enables the kind of plot that is displayed to be changed (see Fig. 2). Users can choose from the following.
“Flow time series” are dynamic plots of observed and simulated discharge as well as precipitation time series and discharge errors.
“Model performance” is an ensemble of static plots of observed and simulated discharge as well as precipitation time series and of annual regimes, flow duration curves, and a scatter plot between simulated and observed discharges.
“State variables” are dynamic plots that show the time series of GR model store levels as well as the time series of internal model fluxes.
“Model diagram” is a plot that can be dynamic and on the right shows the scheme of the chosen GR model and the dynamic evolution of all its fluxes with time and the related hydrometeorological data.
“Select the time window” enables the user to zoom within the provided data period or to move the window using the sliders.
“Select the target date” enables a specific date to be targeted (only for the “Model diagram” panel).
In the center, the plots are proposed by the “Choose a plot” panel.
On the right, the following are found:
A table of criteria provides the values of seven performance criteria (NSE and KGE with use of squared root, inverse, or no transformation of discharge, in addition to the bias).
Using the “Show previous simulation Qold” option, the previously obtained simulated time series appear on the plots provided in the center of the GUI as a dotted gray line. In addition, ticking this option makes criteria of this previous simulation appear in the criteria table introduced above. This option has no effect on the “Model performance” panel.
Two buttons allow users to download the displayed plot in a PNG file format, which can be useful for a report for example (in order to ensure the tracking of the downloaded files, various information is automatically added to the file header: name of the dataset, name of the model, simulation period, and parameter values; see Appendix C), and the hydrometeorological data (including the simulation) in a CSV file format, to be used externally for further analysis or to be saved.
Figure 2 presents the airGRteaching GUI “Modeling” panels (a–d) and “Summary sheet” panels (e–f).
If R is not installed on the students' computers, it is possible to run the
airGRteaching GUI online. Indeed, the graphical user interface is available at https://sunshine.inrae.fr/app/airGRteaching (last access: 20 July 2023) with demo datasets.
2.4.3 Data associated with
Users are free to use their own datasets, but the
airGRteaching package benefits from the
airGRdatasets package (Delaigue et al., 2023a), which contains a dataset of 19 different catchments located in France (Fig. 3 and Table 2). This dataset is a subset of the larger CAMELS-FR dataset (Delaigue et al., 2022) and has been assembled to include various French hydro-climatic regimes, with 12 rain-dominated catchments, 1 rain- and snow-dominated catchment, 2 snow-dominated catchments, 2 Mediterranean catchments, and 2 groundwater-dominated catchments. Figure 4 shows the main characteristics of the catchment set. Catchment area ranges from 25 to 3917 km2, with half of the catchment set draining less than 686 km2.
The dataset is composed of both static geomatic and physiographic catchment indices and hydro-climatic time series (solid and liquid precipitation, potential evapotranspiration, air temperature, and streamflow time series). The climatic time series have been extracted from the SAFRAN reanalysis (Vidal et al., 2010) and aggregated at the catchment scale, while streamflow series have been extracted using the HydroPortail (https://hydro.eaufrance.fr/, last access: 30 December 2022). These hydro-climatic temporal series are available at the daily time step.
This section and the accompanying sections in the Appendix present tests based on the
airGRteaching package and designed to illustrate rainfall-runoff modeling, model calibration, evaluation, and robustness in hydrological classes. These tests are also available as a vignette in the
airGRteaching package: users can thus recreate all these illustrations using their own datasets.
3.1 Understanding rainfall-runoff modeling
3.1.1 The role of model components and parameters
Rainfall-runoff models are composed of different components, e.g., reservoirs or unit hydrographs, whose behavior is defined by equations and parameters. Parameter estimation is a key step toward tailoring the models to a specific catchment. Understanding the role of model components and parameters is therefore an unavoidable preliminary step to performing hydrological modeling.
To illustrate the production and the routing parts of hydrological modeling that are present in any model, it is possible to use the different GR models included in
airGRteaching and to produce rainfall-runoff transformations considering different model parameter values.
The GR4J model (Perrin et al., 2003) comprises a production store (X1 parameter), which determines the actual evapotranspiration and the net rainfall (see Appendix C4 for a GR4J flowchart). The routing of net rainfall is determined through two unit hydrographs (X4 parameter) and a routing store (X3 parameter). A final component, representing the intercatchment groundwater exchange, is determined by the X2 parameter.
As an example, the command lines in Appendix Listing D1 and Fig. 5 illustrate the role of the X2 parameter in the production part of the rainfall-runoff transformation, showing higher streamflow values simulated with higher X2 values, since higher X2 parameter values lead to more positive incoming water from groundwater. Moreover, Fig. 6 illustrates the role of the X4 parameter in the routing part of the rainfall-runoff transformation, with delayed flood peak values when considering higher X4 values (see command lines in Appendix Listing D2).
The relative importance of the production and routing functions depends on the time step considered for the rainfall-runoff simulation. The production process is more important for the larger time steps (e.g., month or year) since it controls the catchment water balance. This can be easily illustrated by aggregating simulations performed at a daily time step to a yearly time step (see command lines in Appendix Listing D3). Figure 7 compares, at the annual time step, the GR4J daily simulations performed using different X2 parameter values with the simulations performed using different X4 parameter sets. We can observe that at the annual time step, the impact of considering different X4 parameter values is limited compared to the use of different X2 parameter values.
3.1.2 On the need to perform a model warm-up
Initial values of the model water storage must be specified at the beginning of a simulation. The way initial levels are defined can lead to potentially significant model errors. The most convenient way for modelers to initialize rainfall-runoff models is to perform a warm-up run of the model in order to limit the impact of this unknown.
This issue can be illustrated with
airGRteaching by considering different warm-up period lengths (see command lines in Appendix Listing D4). Figure 8 illustrates a portion of the streamflow simulations obtained considering (i) no warm-up period, (ii) a 1-month warm-up period, and (iii) a 1-year warm-up period of the two GR4J stores. Figure 8 shows that the three simulations converge after a bit more than 5 months, reinforcing the necessity of performing a sufficiently long warm-up. Please note that by default,
airGRteaching initializes the production and the routing stores at 30 % and 50 % of their capacity, respectively.
3.2 Model calibration, evaluation, and robustness
3.2.1 Manual calibration
airGRteaching GUI (see Fig. 2), it is possible to test different parameter sets of the GR rainfall-runoff models and to estimate the performance of each tested parameter set in order to perform a manual calibration. A classic way to do so through the
airGRteaching GUI is to select a criterion as an objective function in the table showing the criteria values on the right, to activate the “Show previous simulations (Qold)”, and to modify parameter values step by step until the simulation and criterion are satisfactory.
This can also be done using the simple command-line functions (
SimGR(); see command lines in Appendix Listing D5).
3.2.2 Automatic calibration
Automatic calibration of model parameters is also possible in
airGRteaching using the procedure described by Michel (1991) and by considering one objective function such as NSE or KGE. To do so, there are two options in
clicking on the automatic calibration button in the
using the simple
airGRteachingcommand-line functions (
CalGR(); see command lines in Appendix Listing D6).
A systematic inspection of the parameter space is performed to determine the most likely zone of convergence. This is done either by direct grid screening or by constrained sampling based on empirical parameter databases.
From the best parameter set of the previous step, a steepest-descent local search procedure is carried out to find an estimate of the optimum parameter set.
airGRteaching allows the second step of this procedure to be visualized (see command lines in Appendix Fig. A5).
3.2.3 How to evaluate model calibration
Different ways to evaluate the model calibration performance may be conceived using
airGRteaching: evaluating criteria on the calibration period, examining the graphical summary of the calibration performance (
airGR::plot()), and comparing simulated and observed streamflow temporal series, etc.
Analyzing simulated versus observed flow regimes is an informative indicator of model performance (see command lines in Appendix Listing D7). Figure 9 compares regimes in a mountainous catchment (located in the French Alps), while the flow simulation has been obtained with and without taking into account snow accumulation and melt. The regime comparison might be compelling for the students, hopefully leading them to use an additional snow accumulation and melt routine (such as CemaNeige, Valéry et al., 2014, available in
3.2.4 Objective functions for model calibration
Oudin et al. (2006) and other authors showed the impact of using flow transformation in objective functions used for model calibration. It is possible, in
airGRteaching, to apply different flow transformations to the objective function used for model parameter calibration (see command lines in Appendix Listing D8). Figure 10 compares the simulations performed considering GR4J parameter sets obtained after a calibration on (i) NSE calculated on natural flows (denoted as NSEQ hereafter), (ii) NSE calculated on square-root-transformed flows (denoted as NSE hereafter), and (iii) NSE calculated on logarithmic-transformed flows (denoted as NSElog Q hereafter), emphasizing performance in high, mean, and low flows, respectively. Logically, we can observe that the model calibrated on NSEQ performs better for high-flow periods, and the model calibrated on NSElog Q performs better for low-flow periods, while the model calibrated on NSE performs in between.
Similarly to the use of different flow transformations during model calibration, the
CalGR() function allows us to test several objective functions such as NSE or KGE (see command lines in Appendix Listing D9).
3.2.5 Model evaluation and robustness
Split-sample tests, i.e., calibrating and evaluating a model on non-overlapping periods (Klemeš, 1986), is key for the assessment of model transferability in time, since in practice models are used outside their calibration conditions. Split-sample tests can be performed for model calibration and evaluation using both
airGRteaching functions, respectively (see command lines in Appendix Listing D10).
The differential split-sample test, also introduced by Klemeš (1986), consists in identifying two climatically contrasted periods in the available record and performing the split-sample test using these two periods. Table 3 presents the calibration and evaluation performance of the GR4J model obtained for two sub-periods, composed of the wettest and the driest hydrological years (based on the aridity index, i.e., the total annual precipitation divided by the total annual potential evapotranspiration; see command lines in Appendix Listing D11).
The basic manipulations of the
airGRteaching package illustrated in the previous sections can also be used in more comprehensive hydrological teaching projects, presented in a vignette format in the package (example in Fig. 11) available in both English and French. These three projects deal with flow reconstruction (i.e., producing simulated streamflow over periods for which records are missing), flow forecasting (i.e., anticipating streamflow conditions for days ahead from given initial conditions), and climate change applications (i.e., transforming climate projections into hydrological projections). These can be run as stand-alone projects with the dataset available in the
airGRdatasets package or run on other catchments by importing the necessary hydro-climatic series.
Streamflow reconstruction. The Estéron at Broc catchment presents flow observation from 1999 to 2018 but also several missing data in 2004. This project aims to use the hydro-climatic series available and the GR2M model to reconstruct the missing flow data through rainfall-runoff simulation. The concepts addressed and the skills developed with this project are (i) parameter calibration (both manually and automatically) using an objective function and (ii) calibration–evaluation methodology.
Low-flow forecasting. This project aims to use the hydro-climatic data available for the Meuse at Saint-Mihiel catchment and the GR6J rainfall-runoff model to forecast the flows for the autumn of 2018, using (i) the last observed streamflow value, (ii) historical rainfall observations, and (iii) historical flow observations (see Fig. 12). The concepts addressed and the skills developed with this project are (i) the definition of climatology, (ii) flow forecasting, and (iii) flow assimilation.
Impact of climate change on streamflow regime. Using catchment-scale delta-change-derived future climate projections, this project aims at quantifying the impact of climate change on the flow regime of the Durance at Embrun catchment (see Fig. 13). The concepts addressed and the skills developed with this project are the (i) delta-change method, (ii) flow regime, (iii) bias correction, and (iv) impact of snow on the flow regime.
Users of the
airGRteaching package may also produce their own exercises as
airGRteaching vignettes, based on the three examples provided.
We believe that the proposed exercises and projects are a must if one wishes to learn hydrological modeling. They represent the core of many catchment-related studies.
Like any tool,
airGRteaching has its limitations. The first one is that so far only GR hydrological models have been available in
airGRteaching. Adding other models is feasible, but to do so, they should be implemented to be compatible with the
airGR framework (which contains the basic components for
airGRteaching). While for the command-line use of
airGRteaching (i.e., use of the
SimGR() functions), this should be easy to implement, the GUI implementation would require more efforts (for instance, it would require the adding of a model scheme for each model; the interface could become less handy, with models presenting over 10 parameters to optimize, and calibration would be far less rapid).
In addition, it is not possible for the user to build their own hydrological model by adding, for example, reservoirs (e.g., with different discharge functions) and unit hydrographs, to help understand each compartment of a model. This is possible with the
RS MINERVE software (García Hernández et al., 2020).
Other limitations, as mentioned in Sect. 2.3, are that
airGRteaching only offers a limited set of modeling options, compared to
airGR. This however could also be seen as a strength, as proposing too many options could be cumbersome from a user's perspective, and these limitations are therefore voluntary.
Remote sensing data, other than meteorological or hydrological data, cannot be used in
airGRteaching at the moment. In addition, the effect of land cover changes cannot directly be an asset to
airGRteaching, as is the case in some physically based models.
Finally, proper uncertainty exercises, apart from the calibration on different periods, have so far not been a part of this tool, which we see as a simple way of starting hydrological modeling. However, it is easy enough to add noise to the input data to assess input uncertainty. Uncertainty arising from model structure can only be studied by changing models (e.g., using GR4J and GR5J models). The uncertainty associated with parameter calibration methods cannot be tested, as only an optimization algorithm is provided (NB: other algorithms can be plugged into
airGRteaching does not provide turnkey tools for visualizing uncertainty (e.g., error bars or envelopes on streamflow simulation).
Exercises linking hydrology with other disciplines and scientific communities could be developed by the coupling of the
airGRteaching package with other numerical tools and models. First, using actual global (GCM) or regional (RCM) climate model outputs as rainfall-runoff model inputs would illustrate the impact of climate variability or emission scenarios on catchment hydrology, linking climatology and hydrology. In a similar way, streamflows produced by the
airGRteaching package could be used as inputs to hydraulic models to produce flood maps in teaching projects involving both hydrological and hydraulic skills. Finally, coupling
airGRteaching with models of water uses (e.g., water withdrawal models for drinking water or irrigation) would have interesting teaching applications. Another valuable perspective is to use remote sensing data to perform data assimilation for hydrological forecasting by recovering real-time meteorological (e.g., precipitation measured by rain gauges), hydrological (e.g., streamflow observed from gauging stations), or even satellite data (e.g., MODIS snow cover observations) and using these data as inputs of a rainfall-runoff model in the
airGRteaching package, e.g., with the
airGRdatassim package (Piazzi et al., 2021; Piazzi and Delaigue, 2021). Such applications would illustrate the added value of assimilating hydro-meteorological data for better modeling in hydrology. Other exercises could be centered around uncertainties, through coupling the airGRteaching package with sensitivity analysis methods.
airGRteaching package could be used for the development of serious games devoted to hydro-meteorological applications, aiming, for example, to discuss the issues of making better decisions when considering probabilistic forecasts (Ramos et al., 2013).
The authors' experience with different audiences has shown that
airGRteaching is useful in helping students understand a variety of basic concepts: from the choice of an objective function to the sensitivity of model simulations to individual parameters, the difference between model states and model parameters, the difference between automatic and manual calibration, and the informative and complementary value of a variety of plots. Projects that are more elaborate have been developed and are listed in Sect. 4. For students, depending on the time allotted and their experience, we use the graphical interface with or without the use of computer code. For researchers, it is more a matter of introducing them specifically to GR models, and the interface is used as an introduction of the GR model structure. For engineers working in consulting firms, it is often somewhere in between, depending on their experience and their background. The GUI is frequently used to avoid being bogged down in problems of form and to concentrate exclusively on the underlying concepts of hydrological modeling. The simplified code version allows a smooth transition to the more complex
airGR code. For the general public, the aim is usually to introduce them using the
airGRteaching GUI to one of the fields of hydrology, to help them understand what a model is, and to raise their awareness of applications such as flood and low-flow forecasting and global change.
The introduction to computer programming is ideal to teach these notions to students. If students are to take this tool into their own hands, they must gradually acquire the concepts without difficulty. It is therefore essential that this is done in a playful way so that they are not discarded. The use of a graphical interface allowing modeling notions to be acquired, while putting aside the programming aspects, allows the different problems to be separated: modeling on the one hand and programming on the other hand. As soon as they wish to go further in the understanding of their subject, students very quickly perceive the limitations that a graphical interface can represent (options too limited, etc.). In addition, use can quickly become daunting if tasks need to be repeated (for example, clicking a large number of times and in a well-defined order to reproduce the results on several datasets). Sometimes there is not enough time to learn programming, justifying the need to use simple tools.
As such, the
airGRteaching tool is not intended to be used to realize extended hydrological research studies, and therefore it does not aim to be used to contribute to the actual solving of any of the 23 UPHs (unsolved problems in hydrology; Blöschl et al., 2019). However, as it is a tool to teach hydrology, to understand hydrological processes, and to master hydrological modeling, we believe that
airGRteaching could be used as a preliminary step in the solving of some UPHs. Namely, UPH19 (“How can hydrological models be adapted to be able to extrapolate to changing conditions, including changing vegetation dynamics?”) and UPH20 (“How can we disentangle and reduce model structural/parameter/input uncertainty in hydrological prediction?”), due to the many model parameter manipulations and calibration–evaluation exercises that
airGRteaching proposes, are good candidates. This tool can contribute to UPH21 (“How can the (un)certainty in hydrological predictions be communicated to decision makers and the general public?”) as it has already been used by several decision makers in hydrological training.
airGRteaching can be seen as a gateway to mastering
airGR and other
airGR-dependent packages, thus indirectly helping to solve other UPHs. This is notably the case for questions UPH22 (“What are the synergies and tradeoffs between societal goals related to water management (e.g., water-environment-energy-food-health)?”) and UPH23 (“What is the role of water in migration, urbanization, and the dynamics of human civilizations, and what are the implications for contemporary water management?”), linked to water usage, thanks to the
airGRiwrm package (Dorchies et al., 2022), which allows water resources management to be integrated. This package could help to solve problems of spatial heterogeneity and change of scale, namely UPH5 (“What causes spatial heterogeneity and homogeneity in runoff, evaporation, subsurface water and material fluxes (carbon and other nutrients, sediments), and in their sensitivity to their controls (e.g., snowfall regime, aridity, reaction coefficients)?”) and UPH6 (“What are the hydrologic laws at the catchment scale and how do they change with scale?”) because it simplifies the use of
airGR in a semi-distributed mode. The
airGRdatassim package, which enables data assimilation, could be linked to questions of prediction uncertainty, namely UPH20.
Teaching hydrological modeling requires hands-on experience with rainfall-runoff models. Dedicated tools need to be adapted to the skills of the students and users and preferably developed in an open-source programming language to ensure the reproducibility of the results. In this context, the
airGRteaching R package has been developed as an add-on to the
airGR package, which gathers several lumped rainfall-runoff models widely used by hydrological researchers and practitioners.
airGRteaching contains a graphical user interface and allows teachers and students to import their own data and create their own exercises. A specific dataset of 19 different catchments in France is included in the add-on
airGRdatasets package. This dataset is composed of hydro-climatic time series (solid and liquid precipitation, potential evapotranspiration, air temperature, and streamflow time series). Finally, three hydrological teaching projects are proposed aimed at (i) using a monthly rainfall-runoff model to reconstruct flow series, (ii) using a daily model to forecast low flows, and (iii) studying the impact of climate change on streamflow of a mountainous catchment. Thanks to its open nature, other projects may be added to the package by
airGRteaching users, based on the dataset provided or other datasets.
In this appendix, we have used the time series of the X045401001 catchment (the Ubaye at Lauzet-Ubaye [Roche-Rousse] – DREAL PACA). The GR5J model, coupled to CemaNeige, was calibrated on the raw flows of the period from 1 January 2001 to 31 December 2004. The objective function used is the KGE.
In this appendix, we have used the time series of the X045401001 catchment (the Ubaye at Lauzet-Ubaye [Roche-Rousse] – DREAL PACA). The GR5J model, coupled to CemaNeige, was calibrated on the raw flows of the period from 1 January 2001 to 31 December 2004. The objective function used is the KGE.
In this appendix, we have used the time series of the X045401001 catchment (the Ubaye at Lauzet-Ubaye [Roche-Rousse] – DREAL PACA). The GR5J model, coupled to CemaNeige, was calibrated on the raw flows of the period from 1 January 2001 to 31 December 2004. The objective function used is the KGE.
OD, PB, and GT conceptualized the work. All authors contributed to the
airGRteaching package development: LC implemented a first version of the GUI, OD created and maintains the package (added features, improved GUI, and wrote documentation and vignettes), PB coded the model diagram graph and wrote the vignettes containing the exercises, and GT beta-tested the package and provided documentation and code improvements. OD, PB, and GT drafted the manuscript. All authors reviewed and edited the manuscript.
The contact author has declared that none of the authors has any competing interests.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The authors would like to thank Météo-France (https://www.data.gouv.fr/en/organizations/meteo-france/, last access: 30 December 2022) and the SCHAPI (https://hydro.eaufrance.fr/, last access: 30 December 2022) for providing the SAFRAN meteorological series and the streamflow series included in the
airGRdatasets package. We would like to thank Charles Perrin and Vazken Andréassian for their useful remarks and for their suggestions that significantly improved the paper and the package vignettes. We thank Léo Carriba Demange, Ayoub Chanoual, and Antoine Gazull for their student report on the evaluation of the modeling software available for teaching hydrological modeling. Finally, we thank Matjaz Mikos (the editor) and the anonymous referees for their comments that improved the paper.
This paper was edited by Matjaz Mikos and reviewed by two anonymous referees.
Addor, N., Newman, A. J., Mizukami, N., and Clark, M. P.: The CAMELS data set: catchment attributes and meteorology for large-sample studies, Hydrol. Earth Syst. Sci., 21, 5293–5313, https://doi.org/10.5194/hess-21-5293-2017, 2017. a
AghaKouchak, A., Nakhjiri, N., and Habib, E.: An educational model for ensemble streamflow simulation and uncertainty analysis, Hydrol. Earth Syst. Sci., 17, 445–452, https://doi.org/10.5194/hess-17-445-2013, 2013. a
Astagneau, P. C., Thirel, G., Delaigue, O., Guillaume, J. H. A., Parajka, J., Brauer, C. C., Viglione, A., Buytaert, W., and Beven, K. J.: Technical note: Hydrology modelling R packages – a unified analysis of models and practicalities from a user perspective, Hydrol. Earth Syst. Sci., 25, 3937–3973, https://doi.org/10.5194/hess-25-3937-2021, 2021. a, b
Baahmed, D., Oudin, L., and Errih, M.: Current runoff variations in the Macta catchment (Algeria): is climate the sole factor? [Le facteur climatique est-il la seule cause des modifications actuelles de l'écoulement dans le bassin versant de la Macta (Algérie)?], Hydrolog. Sci. J., 60, 1331–1339, https://doi.org/10.1080/02626667.2014.975708, 2015. a
Belarbi, H., Touaibia, B., Boumechra, N., Amiar, S., and Baghli, N.: Drought and modification of the rainfall-runoff relation: case of Wadi Sebdou basin (western Algeria) [Sécheresse et modification de la relation pluie–débit: cas du bassin versant de l'Oued Sebdou (Algérie Occidentale)], Hydrolog. Sci. J., 62, 124–136, https://doi.org/10.1080/02626667.2015.1112394, 2017. a
Bezak, N., Jemec Auflič, M., and Mikoš, M.: Application of hydrological modelling for temporal prediction of rainfall-induced shallow landslides, Landslides, 16, 1273–1283, https://doi.org/10.1007/s10346-019-01169-9, 2019. a
Blöschl, G., Bierkens, M. F. P., Chambel, A., Cudennec, C., Destouni, G., Fiori, A., Kirchner, J. W., McDonnell, J. J., Savenije, H. H. G., Sivapalan, M., Stumpp, C., Toth, E., Volpi, E., Carr, G., Lupton, C., Salinas, J., Széles, B., Viglione, A., Aksoy, H., et al.: Twenty-three unsolved problems in hydrology (UPH) – a community perspective, Hydrolog. Sci. J., 64, 1141–1158, https://doi.org/10.1080/02626667.2019.1620507, 2019. a
Brigode, P., Lilas, D., Andréassian, V., Nicolle, P., Le Moine, N., Perrin, C., Gremminger, S., and Augeard, B.: Une cartographie de l'écoulement des rivières de Corse, La Houille Blanche, 1, 68–77, https://doi.org/10.1051/lhb/2019009, 2019. a
Brigode, P., Génot, B., Lobligeois, F., and Delaigue, O.: Summary sheets of watershed-scale hydroclimatic observed data for France, Recherche Data Gouv [data set], https://doi.org/10.15454/UV01P1, 2020. a
Burt, T. and Butcher, D.: Stimulation from simulation? A teaching model of hillslope hydrology for use on microcomputers, J. Geogr. High. Educ., 10, 23–39, https://doi.org/10.1080/03098268608708953, 1986. a
Carriba Demange, L., Chanoual, A., and Gazull, A.: Evaluation des logiciels, modèles et packages disponibles pour l'enseignement de la modélisation hydrologique, Projet d'ingénierie GE5, Polytech Nice Sophia, Université Côte d'Azur, https://hal.science/hal-04191446 (last access: 20 July 2023), 2022. a
Cassagnole, M., Ramos, M.-H., Zalachori, I., Thirel, G., Garçon, R., Gailhard, J., and Ouillon, T.: Impact of the quality of hydrological forecasts on the management and revenue of hydroelectric reservoirs – a conceptual approach, Hydrology and Earth System Sciences, 25, 1033–1052, https://doi.org/10.5194/hess-25-1033-2021, 2021. a
Ceola, S., Arheimer, B., Baratti, E., Blöschl, G., Capell, R., Castellarin, A., Freer, J., Han, D., Hrachowitz, M., Hundecha, Y., Hutton, C., Lindström, G., Montanari, A., Nijzink, R., Parajka, J., Toth, E., Viglione, A., and Wagener, T.: Virtual laboratories: new opportunities for collaborative water science, Hydrol. Earth Syst. Sci., 19, 2101–2117, https://doi.org/10.5194/hess-19-2101-2015, 2015. a
Chang, W., Cheng, J., Allaire, J., Sievert, C., Schloerke, B., Xie, Y., Allen, J., McPherson, J., Dipert, A., and Borges, B.: shiny: Web Application Framework for R, R package version 1.7.2, https://CRAN.R-project.org/package=shiny (last access: 20 July 2023), 2022. a
Chauveau, M., Chazot, S., Perrin, C., Bourgin, P.-Y., Sauquet, E., Vidal, J.-P., Rouchy, N., Martin, E., David, J., Norotte, T., Maugis, P., and De Lacaze, X.: Quels impacts des changements climatiques sur les eaux de surface en France à l´horizon 2070?, La Houille Blanche, 4, 5–15, https://doi.org/10.1051/lhb/2013027, 2013. a
Clark, M. P., Kavetski, D., and Fenicia, F.: Pursuing the method of multiple working hypotheses for hydrological modeling, Water Resour. Res., 47, W09301, https://doi.org/10.1029/2010WR009827, 2011. a
Coron, L., Thirel, G., Delaigue, O., Perrin, C., and Andréassian, V.: The Suite of Lumped GR Hydrological Models in an R package, Environ. Model. Softw., 94, 166–171, https://doi.org/10.1016/j.envsoft.2017.05.002, 2017. a, b, c, d, e, f
Coron, L., Delaigue, O., Thirel, G., Dorchies, D., Perrin, C., and Michel, C.: airGR: Suite of GR Hydrological Models for Precipitation-Runoff Modelling, R package version 1.7.0, https://doi.org/10.15454/EX11NA, https://CRAN.R-project.org/package=airGR (last access: 5 August 2023), 2022. a, b, c
Delaigue, O., Thirel, G., Coron, L., and Brigode, P.: airGR and airGRteaching: Two Open-Source Tools for Rainfall-Runoff Modeling and Teaching Hydrology, in: HIC 2018, 13th International Conference on Hydroinformatics, vol. 3 of EPiC Series in Engineering, edited by: La Loggia, G., Freni, G., Puleo, V., and De Marchis, M., EasyChair, 541–548, https://doi.org/10.29007/qsqj, 2018. a
Delaigue, O., Brigode, P., Andréassian, V., Perrin, C., Etchevers, P., Soubeyroux, J.-M., Janet, B., and Addor, N.: CAMELS-FR: A large sample hydroclimatic dataset for France to explore hydrological diversity and support model benchmarking, https://hal.inrae.fr/hal-03687235 (last access: 30 December 2022), 2022. a, b
Delaigue, O., Brigode, P., and Thirel, G.: airGRdatasets: Hydro-Meteorological Catchments Datasets for the “airGR” Packages, R package version 0.2.1, https://doi.org/10.57745/3SPJ4B, https://CRAN.R-project.org/package=airGRdatasets (last access: 5 August 2023), 2023a. a, b
Delaigue, O., Coron, L., Brigode, P., and Thirel, G.: airGRteaching: Teaching Hydrological Modelling with GR (Shiny Interface Included), R package version 0.3.2, https://doi.org/10.15454/W0SSKT, https://CRAN.R-project.org/package=airGRteaching (last access: 5 August 2023), 2023b. a, b
de Lavenne, A., Andréassian, V., Thirel, G., Ramos, M.-H., and Perrin, C.: A Regularization Approach to Improve the Sequential Calibration of a Semidistributed Hydrological Model, Water Resour. Res., 55, 8821–8839, https://doi.org/10.1029/2018WR024266, 2019. a
Desclaux, T., Lemonnier, H., Genthon, P., Soulard, B., and Gendre, R. L.: Suitability of a lumped rainfall–runoff model for flashy tropical watersheds in New Caledonia, Hydrolog. Sci. J., 63, 1689–1706, https://doi.org/10.1080/02626667.2018.1523613, 2018. a
Dorchies, D., Thirel, G., Jay-Allemand, M., Chauveau, M., Dehay, F., Bourgin, P.-Y., Perrin, C., Jost, C., Rizzoli, J.-L., Demerliac, S., and Thépot, R.: Climate change impacts on multi-objective reservoir management: case study on the Seine River basin, France, Int. J. River Basin Manage., 12, 265–283, https://doi.org/10.1080/15715124.2013.865636, 2014. a
Dorchies, D., Delaigue, O., and Thirel, G.: airGRiwrm: “airGR” Integrated Water Resource Management, R package version 0.6.1, https://doi.org/10.15454/3CVD1I, https://CRAN.R-project.org/package=airGRiwrm (last access: 5 August 2023), 2022. a
Elshorbagy, A.: Learner-centered approach to teaching watershed hydrology using system dynamics, Int. J. Eng. Educ., 21, 1203–1213, 2005. a
Ficchì, A., Perrin, C., and Andréassian, V.: Hydrological modelling at multiple sub-daily time steps: Model improvement via flux-matching, J. Hydrol., 575, 1308–1327, https://doi.org/10.1016/j.jhydrol.2019.05.084, 2019. a
Fiering, M. B.: Streamflow Synthesis, Harvard University Press, Cambridge, Mass., ISBN 9780674189270, 1967. a
Fuka, D., Walter, M., Archibald, J., Steenhuis, T., and Easton, Z.: EcoHydRology: A Community Modeling Foundation for Eco-Hydrology, R package version 0.4.12.1, CRAN, https://CRAN.R-project.org/package=EcoHydRology (last access: 20 July 2023), 2018. a
Furusho, C., Perrin, C., Viatgé, J., Lamblin, R., and Andréassian, V.: Synergies entre acteurs opérationnels et scientifiques au service de l'amélioration de la prévision des crues, La Houille Blanche, 4, 5–10, https://doi.org/10.1051/lhb/2016033, 2016. a
García Hernández, J., Paredes Arquiola, J., Foehn, A., Roquier, B., and Fluixá-Sanmartín, J.: RS MINERVE – Technical Manual v2.25, Tech. rep., RS MINERVE Group, Sion, Switzerland, https://crealp.ch/wp-content/uploads/2021/09/rsminerve_technical_manual_v2.25.pdf (last access: 30 August 2023), 2020. a, b
GEBCO Bathymetric Compilation Group 2021: The GEBCO_2021 Grid – a continuous terrain model of the global oceans and land, ERC EDS British Oceanographic Data Centre NOC [data set], https://doi.org/10.5285/c6612cbe-50b3-0cff-e053-6c86abc09f8f, 2021. a
Gupta, H. V., Kling, H., Yilmaz, K. K., and Martinez, G. F.: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling, J. Hydrol., 377, 80–91, https://doi.org/10.1016/j.jhydrol.2009.08.003, 2009. a
Hall, C. A., Saia, S. M., Popp, A. L., Dogulu, N., Schymanski, S. J., Drost, N., van Emmerik, T., and Hut, R.: A hydrologist's guide to open science, Hydrol. Earth Syst. Sci., 26, 647–664, https://doi.org/10.5194/hess-26-647-2022, 2022. a
Hutton, C., Wagener, T., Freer, J., Han, D., Duffy, C., and Arheimer, B.: Most computational hydrology is not reproducible, so is it really science?, Water Resour. Res., 52, 7548–7555, https://doi.org/10.1002/2016WR019285, 2016. a, b
Irving, K., Kuemmerlen, M., Kiesel, J., Kakouei, K., Domisch, S., and Jähnig, S. C.: A high-resolution streamflow and hydrological metrics dataset for ecological modeling using a regression model, Sci. Data, 5, 180224, https://doi.org/10.1038/sdata.2018.224, 2018. a
Kay, D., Kay, N., and McDonald, A.: Teaching Catchment Hydrology: Two Dynamic Models for Classroom Use, Teach. Geogr., 7, 118–124, 1982. a
Kling, H., Fuchs, M., and Paulin, M.: Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios, J. Hydrol., 424–425, 264–277, https://doi.org/10.1016/j.jhydrol.2012.01.011, 2012. a
Knoben, W. J. M. and Spieler, D.: Teaching hydrological modelling: illustrating model structure uncertainty with a ready-to-use computational exercise, Hydrol. Earth Syst. Sci., 26, 3299–3314, https://doi.org/10.5194/hess-26-3299-2022, 2022. a
Kouassi, A., Koffi, Y., Kouame, K., Lasm, T., and Biemi, J.: Modeling of annual flows using a conceptual model and an artificial neural network model in the N'zi-Bandama watershed (Côte d'Ivoire), Agris On-line Papers in Economics and Informatics, 2, 2082–2094, 2012. a
Lehner, B. and Grill, G.: Global river hydrography and network routing: baseline data and new approaches to study the world's large river systems, Hydrol. Process., 27, 2171–2186, https://doi.org/10.1002/hyp.9740, 2013. a
Le Moine, N.: Le bassin versant de surface vu par le souterrain: une voie d’amélioration des performances et du réalisme des modèles pluie-débit?, PhD thesis, Université Pierre et Marie Curie, Paris 6, https://hal.science/tel-02591478 (last access: 30 December 2022), 2008. a
Marchane, A., Tramblay, Y., Hanich, L., Ruelland, D., and Jarlan, L.: Climate change impacts on surface water resources in the Rheraya catchment (High Atlas, Morocco), Hydrolog. Sci. J., 62, 979–995, https://doi.org/10.1080/02626667.2017.1283042, 2017. a
Marshall, J. A., Castillo, A. J., and Cardenas, M. B.: The Effect of Modeling and Visualization Resources on Student Understanding of Physical Hydrology, J. Geosci. Educ., 63, 127–139, https://doi.org/10.5408/14-057.1, 2015. a
Martel, J.-L., Demeester, K., Brissette, F., Poulin, A., and Arsenault, R.: HMETS – A simple and efficient hydrology model for teaching hydrological modelling, flow forecasting and climate change impacts, Int. J. Eng. Educ., 33, 1307–1316, 2017. a
Mathevet, T.: Quels modèles pluie-débit globaux au pas de temps horaire? Développements empiriques et comparaison de modèles sur un large échantillon de bassins versants, PhD thesis, ENGREF, Paris, https://hal.science/tel-02587642v1 (last access: 30 December 2022), 2005. a
McConnell, S.: Code complete, 2nd Edn., Microsoft Press, Redmond, Wash. ISBN-13 9780735619678, 2004. a
Merwade, V. and Ruddell, B. L.: Moving university hydrology education forward with community-based geoinformatics, data and modeling resources, Hydrol. Earth Syst. Sci., 16, 2393–2404, https://doi.org/10.5194/hess-16-2393-2012, 2012. a
Michel, C.: Hydrologie appliquée aux petits bassins ruraux, Cemagref, Antony, https://belinrae.inrae.fr/index.php?lvl=notice_display&id=225112 (last access: 1 August 2023), 1991. a
Mouelhi, S.: Vers une chaîne cohérente de modèles pluie-débit conceptuels globaux aux pas de temps pluriannuel, annuel, mensuel et journalier, PhD thesis, Paris, ENGREF, https://hal.science/tel-00005696v1 (last access: 30 December 2022), 2003. a, b
Mouelhi, S., Michel, C., Perrin, C., and Andréassian, V.: Linking stream flow to rainfall at the annual time step: The Manabe bucket model revisited, J. Hydrol., 328, 283–296, https://doi.org/10.1016/j.jhydrol.2005.12.022, 2006a. a
Mouelhi, S., Michel, C., Perrin, C., and Andréassian, V.: Stepwise development of a two-parameter monthly water balance model, J. Hydrol., 318, 200–214, https://doi.org/10.1016/j.jhydrol.2005.06.014, 2006b. a
Neumann, J. L., Arnal, L., Emerton, R. E., Griffith, H., Hyslop, S., Theofanidi, S., and Cloke, H. L.: Can seasonal hydrological forecasts inform local decisions and actions? A decision-making activity, Geosci. Commun., 1, 35–57, https://doi.org/10.5194/gc-1-35-2018, 2018. a
Nicolle, P., Pushpalatha, R., Perrin, C., François, D., Thiéry, D., Mathevet, T., Le Lay, M., Besson, F., Soubeyroux, J.-M., Viel, C., Regimbeau, F., Andréassian, V., Maugis, P., Augeard, B., and Morice, E.: Benchmarking hydrological models for low-flow simulation and forecasting on French catchments, Hydrol. Earth Syst. Sci., 18, 2829–2857, https://doi.org/10.5194/hess-18-2829-2014, 2014. a
Oudin, L., Andréassian, V., Mathevet, T., Perrin, C., and Michel, C.: Dynamic averaging of rainfall-runoff model simulations from complementary model parameterizations, Water Resour. Res., 42, W07410, https://doi.org/10.1029/2005WR004636, 2006. a
Paquet, E., Garavaglia, F., Garçon, R., and Gailhard, J.: The SCHADEX method: A semi-continuous rainfall–runoff simulation for extreme flood estimation, J. Hydrol., 495, 23–37, https://doi.org/10.1016/j.jhydrol.2013.04.045, 2013. a
Pérez-Sánchez, J., Senent-Aparicio, J., and Jimeno-Sáez, P.: The application of spreadsheets for teaching hydrological modeling and climate change impacts on streamflow, Comput. Appl. Eng. Educ., 30, 1510–1525, https://doi.org/10.1002/cae.22541, 2022. a
Piazzi, G. and Delaigue, O.: airGRdatassim: Suite of Tools to Perform Ensemble-Based Data Assimilation in GR Hydrological Models, R package version 0.1.3, https://doi.org/10.15454/WEYYVZ, https://CRAN.R-project.org/package=airGRdatassim (last access: 5 August 2023) 2021. a
Piazzi, G., Thirel, G., Perrin, C., and Delaigue, O.: Sequential Data Assimilation for Streamflow Forecasting: Assessing the Sensitivity to Uncertainties and Updated Variables of a Conceptual Hydrological Model at Basin Scale, Water Resour. Res., 57, e2020WR02839, https://doi.org/10.1029/2020WR028390, 2021. a
Pushpalatha, R., Perrin, C., Le Moine, N., Mathevet, T., and Andréassian, V.: A downward structural sensitivity analysis of hydrological models to improve low-flow simulation, J. Hydrol, 411, 66–76, https://doi.org/10.1016/j.jhydrol.2011.09.034, 2011. a
Riboust, P., Thirel, G., Moine, N. L., and Ribstein, P.: Revisiting a Simple Degree-Day Model for Integrating Satellite Data: Implementation of Swe-Sca Hystereses, J. Hydrol. Hydromech., 67, 70–81, https://doi.org/10.2478/johh-2018-0004, 2019. a, b
Richmond, B., Aspinwall, D., Vescuso, P., Peterson, S., and High Performance Systems, Inc.: STELLA, High Performance, Lyme, NH, OCLC: 14639320, https://www.iseesystems.com (last access: 1 August 2023), 1985. a
Roux, Q. and Brigode, P.: How long would we have to wait before (re)filling the Malpasset dam reservoir? An example of a teaching project done using R and airGR modeling packages, https://hal.science/hal-03020769 (last access: 20 July 2023), 2018. a
Sanchez, C. A., Ruddell, B. L., Schiesser, R., and Merwade, V.: Enhancing the T-shaped learning profile when teaching hydrology using data, modeling, and visualization activities, Hydrol. Earth Syst. Sci., 20, 1289–1299, https://doi.org/10.5194/hess-20-1289-2016, 2016. a
Santos, L., Thirel, G., and Perrin, C.: Technical note: Pitfalls in using log-transformed flows within the KGE criterion, Hydrol. Earth Syst. Sci., 22, 4583–4591, https://doi.org/10.5194/hess-22-4583-2018, 2018. a
Seibert, J. and Vis, M. J. P.: Teaching hydrological modeling with a user-friendly catchment-runoff-model software package, Hydrol. Earth Syst. Sci., 16, 3315–3325, https://doi.org/10.5194/hess-16-3315-2012, 2012. a
Slater, L. J., Thirel, G., Harrigan, S., Delaigue, O., Hurley, A., Khouakhi, A., Prosdocimi, I., Vitolo, C., and Smith, K.: Using R in hydrology: a review of recent developments and future directions, Hydrol. Earth Syst. Sci., 23, 2939–2963, https://doi.org/10.5194/hess-23-2939-2019, 2019. a, b, c
Tarboton, D., Idaszak, R., Horsburgh, J., Heard, J., Ames, D., Goodall, J., Band, L., Merwade, V., Couch, A., Arrigo, J., Hooper, R., Valentine, D., and Maidment, D.: HydroShare: Advancing Collaboration through Hydrologic Data and Model Sharing, in:7th International Congress on Environmental Modelling and Software - San Diego, California, USA, 15–19 June 2014, https://scholarsarchive.byu.edu/iemssconference/2014/Stream-A/7 (last access: 20 July 2023), 2014. a
Toum, E., Masiokas, M. H., Villalba, R., Pitte, P., and Ruiz, L.: The HBV.IANIGLA Hydrological Model, R J., 13, 378–395, 2021. a
Valéry, A., Andréassian, V., and Perrin, C.: `As simple as possible but not simpler': what is useful in a temperature-based snow-accounting routine? Part 2 – Sensitivity analysis of the Cemaneige snow accounting routine on 380 catchments, J. Hydrol., 517, 1176–1187, https://doi.org/10.1016/j.jhydrol.2014.04.058, 2014. a, b
Vanderkam, D., Allaire, J., Owen, J., Gromer, D., and Thieurmel, B.: dygraphs: Interface to 'Dygraphs' Interactive Time Series Charting Library, R package version 22.214.171.124, https://CRAN.R-project.org/package=dygraphs (last access: 20 July 2023), 2018. a
Vidal, J., Martin, E., Franchistéguy, L., Baillon, M., and Soubeyroux, J.: A 50-year high-resolution atmospheric reanalysis over France with the Safran system, Int. J. Climatol., 30, 1627–1644, https://doi.org/10.1002/joc.2003, 2010. a
Viglione, A. and Parajka, J.: TUWmodel: Lumped/Semi-Distributed Hydrological Model for Education Purposes, R package version 1.1-1, https://CRAN.R-project.org/package=TUWmodel (last access: 20 July 2023), 2020. a, b
Wi, S., Ray, P., Demaria, E. M. C., Steinschneider, S., and Brown, C.: A user-friendly software package for VIC hydrologic model development, Environ. Model. Softw., 98, 35–53, https://doi.org/10.1016/j.envsoft.2017.09.006, 2017. a
Zimmerman, W. B. J.: Multiphysics Modeling with Finite Element Methods, in: vol. 18 of eries on Stability, Vibration and Control of Systems, Series A, World Scientific, https://doi.org/10.1142/6141, 2006. a