Articles | Volume 21, issue 1
Hydrol. Earth Syst. Sci., 21, 423–440, 2017
Hydrol. Earth Syst. Sci., 21, 423–440, 2017
Research article
25 Jan 2017
Research article | 25 Jan 2017

Looking beyond general metrics for model comparison – lessons from an international model intercomparison study

Tanja de Boer-Euser1, Laurène Bouaziz2, Jan De Niel3, Claudia Brauer4, Benjamin Dewals5, Gilles Drogue6, Fabrizio Fenicia7, Benjamin Grelier6, Jiri Nossent8,9, Fernando Pereira8, Hubert Savenije1, Guillaume Thirel10, and Patrick Willems3,9 Tanja de Boer-Euser et al.
  • 1Water Resources Section, Faculty of Civil Engineering and Geosciences, Delft University of Technology, P.O. Box 5048, 2600 GA Delft, the Netherlands
  • 2Department Catchment and Urban Hydrology, Deltares, Boussinesqweg 1, 2629 HV Delft, the Netherlands
  • 3Hydraulics division, Department of Civil Engineering, KU Leuven, Kasteelpark Arenberg 40, 3001 Leuven, Belgium
  • 4Hydrology and Quantitative Water Management Group, Wageningen University and Research, P.O. Box 47, 6700 AA Wageningen, the Netherlands
  • 5University of Liège, Place du 20-Août 7, 4000 Liège, Belgium
  • 6LOTERR-UFR SHS, Université de Lorraine, Île du Saulcy, 57045 Metz CEDEX 1, France
  • 7Eawag, Überlandstrasse 133, 8600 Dübendorf, Switzerland
  • 8Flanders Hydraulics Research, Berchemlei 115, 2140 Antwerp, Belgium
  • 9Vrije Universiteit Brussel (VUB), Department of Hydrology and Hydraulic Engineering, Boulevard de la Plaine 2, 1050 Brussels, Belgium
  • 10Irstea, Hydrosystems and Bioprocesses Research Unit (HBAN), 1, rue Pierre-Gilles de Gennes, CS 10030, 92761 Antony CEDEX, France

Abstract. International collaboration between research institutes and universities is a promising way to reach consensus on hydrological model development. Although model comparison studies are very valuable for international cooperation, they do often not lead to very clear new insights regarding the relevance of the modelled processes. We hypothesise that this is partly caused by model complexity and the comparison methods used, which focus too much on a good overall performance instead of focusing on a variety of specific events. In this study, we use an approach that focuses on the evaluation of specific events and characteristics. Eight international research groups calibrated their hourly model on the Ourthe catchment in Belgium and carried out a validation in time for the Ourthe catchment and a validation in space for nested and neighbouring catchments. The same protocol was followed for each model and an ensemble of best-performing parameter sets was selected. Although the models showed similar performances based on general metrics (i.e. the Nash–Sutcliffe efficiency), clear differences could be observed for specific events. We analysed the hydrographs of these specific events and conducted three types of statistical analyses on the entire time series: cumulative discharges, empirical extreme value distribution of the peak flows and flow duration curves for low flows. The results illustrate the relevance of including a very quick flow reservoir preceding the root zone storage to model peaks during low flows and including a slow reservoir in parallel with the fast reservoir to model the recession for the studied catchments. This intercomparison enhanced the understanding of the hydrological functioning of the catchment, in particular for low flows, and enabled to identify present knowledge gaps for other parts of the hydrograph. Above all, it helped to evaluate each model against a set of alternative models.

Short summary
In this study, the rainfall–runoff models of eight international research groups were compared for a set of subcatchments of the Meuse basin to investigate the influence of certain model components on the modelled discharge. Although the models showed similar performances based on general metrics, clear differences could be observed for specific events. The differences during drier conditions could indeed be linked to differences in model structures.