The goal of this study is to evaluate the ERA5 reanalysis product as a substitute for observed data and to compare its properties to those of the older ERA-Interim reanalysis for hydrological modelling uses. Therefore, the ERA5, ERA-Interim and observed (weather station) meteorological datasets were used and basin-averaged over 3138 catchments over Canada and the United States, whose locations and average elevations are shown in Fig. 1. It can be seen that there is a good coverage of the entire domain, although some sparsely populated areas in northern Canada and in the United States Midwest have a lower density of hydrometric gauges.

The hydrological models used in this study required minimum and maximum daily temperature as well as daily precipitation amounts. ERA-Interim and the observed datasets were already on a daily time step; however, ERA5 is an hourly product and, as such, it was necessary to derive daily values from the hourly data by summing precipitations and taking the maximum and minimum 1 h temperatures of the day.

In the course of this study, two lumped hydrological models were implemented and calibrated over each of the available catchments because the large-scale aspect of this study precluded the widespread implementation of distributed models. Although ERA5's spatial resolution is more refined than ERA-Interim (31 km vs. 79 km), it is still coarse enough that a distributed model would not have changed the results dramatically in this regard. The two hydrological models selected to evaluate the performance of the various climate datasets, GR4J and HMETS, are flexible and adaptable and have been shown to perform well in a wide range of climates and hydrological regimes (Arsenault al., 2015, 2018; Martel et al., 2017; Valery et al., 2014; Perrin et al., 2003). It was decided to perform the study using two hydrological models in order to assess the impacts of the climate data selection on the overall uncertainty of the hydrological modelling simulations.

As will be detailed in the following section, the three precipitation and three temperature datasets were combined in their nine possible arrangements for analysis purposes. It follows that the sheer number of calibrations to be performed (3 precipitation datasets × 3 temperature datasets × 2 hydrological models × 3138 catchments) in this study required implementation of automatic model parameter calibration methods. For this study, the CMAES algorithm was implemented because of its flexibility (Hansen, et al., 2003). Indeed, it performs well for small and large parameter spaces such as the 6-parameter and 21-parameter spaces in this study. It was also shown to be robust and is considered to be one of the best auto-calibration algorithms for hydrological modelling (Arsenault et al., 2013).

The hydrological model parameters were calibrated on the entire available record of data for each catchment, foregoing the usual model validation step. This method was chosen for two reasons. First, calibrating on all years ensures that the maximum amount of information from the climate data is present in the parameter set and thus that there is no added uncertainty from choosing calibration and validation years. Second, Arsenault et al. (2018) have shown that the model performance is statistically better when more years are added to the dataset and that validation and calibration skills are not necessarily correlated.

Finally, the calibration objective function was the Kling–Gupta efficiency (KGE) metric, which is a modified version of the Nash–Sutcliffe efficiency metric that was introduced by Gupta et al. (2009) and Kling et al. (2012). KGE corrects the fact that NSE underestimates variability in the goodness-of-fit function. It is defined as a combination of three elements: 1KGE=1-r-12+β-12+γ-12, where r is the correlation component represented by Pearson's correlation coefficient, β is the bias component represented by the ratio of estimated and observed means, and γ is the variability component represented by the ratio of the estimated and observed coefficients of variation.

A perfect fit between observed and simulated flows will return a KGE of 1. Using the mean hydrograph as a predictor returns a KGE of 0, and a KGE lower than 0 implies that the simulated streamflow is a worse predictor of the observed flows than taking the mean of the observed values. KGE values above 0.6 are generally considered good; however, this is a subjective quantification of the quality of the goodness of fit.

The next steps following the calibration of the hydrological models on the 3138 catchments were to analyse the raw climate data (precipitation and temperature) at the catchment scale. This analysis was performed by generating the nine possible arrangements of three precipitation and three temperature datasets and comparing their relative differences. Then, after performing the model calibration and hydrological simulation steps, the same type of comparison was performed using the calibration KGE metric as a proxy to the quality of the climate dataset. For example, if a certain combination of precipitation and temperature datasets generates higher KGE calibration scores, it is assumed that the climate data are more likely to be accurate than another dataset that returns lower KGE scores.

The various analyses were conducted on the yearly scale as well as for the winter (December, January and February, or DJF) and summer (June, July and August, or JJA) seasons. The results were then analysed according to their respective catchment locations, climates and sizes in an effort to explain any relationships or differences between the dataset characteristics (i.e. resolution, physics) and their performance (i.e. KGE scores).

The first part of the study was to compare precipitation and temperature values averaged at the catchment scale. Figure 2 shows the mean annual temperatures for the observations and the ERA5 and ERA-Interim reanalysis products for the catchments in this study (top row). It also shows the mean absolute differences between the datasets for the winter (centre row) and summer seasons (bottom row).

The results in Fig. 2 are averaged at the catchment scale in order to preserve the consistency between the climate data and the hydrological modelling results presented further in this paper. It can be seen that the ERA-Interim and ERA5 temperatures are generally similar to the observations, although ERA-Interim displays a warm bias almost everywhere except for the south-eastern United States and a few catchments in Canada, where it has a cold bias.

On the other hand, ERA5 sees a strong reduction in biases compared to those in the ERA-Interim dataset. The western coast of North America clearly still shows some important biases of up to 3 ∘C in summer and -2 ∘C in summer, although for most catchments the bias amplitude is smaller. It should be noted that most of the large biases are observed in mountainous areas, where observation networks are generally considered less robust. In the panels representing the differences between ERA5 and ERA-Interim in Fig. 2, it can be seen that the ERA5 product corrects the biases in ERA-Interim; i.e. the areas that were too hot in ERA-Interim are colder in ERA5 and vice versa. The south-eastern USA was particularly problematic for ERA-Interim in the context of hydrological modelling (Essou et al., 2016b), and it will therefore be explored further with ERA5 in the rest of this study.

The precipitation time series from the three datasets in this study were compared in a similar manner to the temperature data, with Fig. 3 showing the mean annual precipitation for the observations and the ERA5 and ERA-Interim reanalysis products for the catchments in this study (top row). Figure 3 also shows the mean absolute differences between the datasets for the winter (centre row) and summer seasons (bottom row).

From Fig. 3, it is clear that there is a good representation of mean seasonal and annual precipitation values across the study domain. For winter, it seems that ERA-Interim and ERA5 are very similar, as the differences between those datasets are small. One exception is the western coast, where a dry bias persists although it has been reduced in ERA5 as compared to ERA-Interim. For the summer period, there is a strong reduction in biases for the eastern half of the United States, where ERA-Interim was problematic. The dry/wet bias pattern of ERA-Interim is strongly reduced in ERA5. However, both reanalysis products are wet in the north, although as will be discussed in Sect. 5.1, this might be related to the quality of the observation datasets in the remote northern catchments.

The first results obtained in the hydrological modelling portion of this study were the performance of the hydrological models in calibration when driven by the various combinations of precipitation and temperature data. Figure 4 shows the calibration KGE scores for the HMETS (panel a) and GR4JCN (panel b) for the nine combinations of precipitation (three sets) and temperature (three sets). Each boxplot in Fig. 4 contains the KGE scores of all of the catchments in this study.

From Fig. 4, it seems clear that the observations remain the best source of precipitation data for hydrological modelling. It is clear that for hydrological modelling, the ERA5 dataset is a net improvement over the ERA-Interim reanalysis, ranking second after the observations. For the catchments in this study, using ERA5 precipitation allows reduction of the median gap between the older ERA-Interim reanalysis and the observations by approximately 40 %. The precipitation data are the main driver behind the differences observed between the datasets, as it can also be seen that the variability linked to the temperature dataset is minimal.

Regarding temperature, ERA5 and the observations provide very similar results, whereas ERA-Interim temperature lags slightly behind. In this sense, the temperature data from ERA5 are marginally more accurate for hydrological modelling at the catchment scale than ERA-Interim and are similar to that of the observed temperature dataset.

From Fig. 4, it is also interesting to note that the hydrological models respond similarly to the various inputs, indicating that the improvements seen with ERA5 are due to the dataset rather than the choice of hydrological model. In general, it can also be seen that HMETS performs better than GR4JCN when using the reanalysis datasets (with a median 0.04 KGE improvement), which is modest but statistically significant using a Kruskal–Wallis non-parametric test. HMETS and GR4JCN are statistically equivalent in terms of KGE when using the observed meteorological data.

The hydrological modelling KGE metrics were next analysed with respect to the catchment locations, as seen in Figs. 5 and 6. Figure 5 presents absolute values of KGE metrics for all three datasets and both hydrological models. The differences between hydrological models (first vs. second row) are generally small, although the better performance of HMETS is particularly clear over the Rocky Mountains, and especially in the case of both reanalyses. Both hydrological models perform similarly when using observations as inputs compared to reanalysis.

Focusing on the best performing hydrological model results (first row), two major observations can be made. First, hydrological modelling with observations is clearly superior to using both reanalysis datasets for the eastern part of the US, but not so much for the western US and Canada. Second, hydrological modelling performance using ERA5 appears to be consistently superior to ERA-I. To better emphasize these conclusions, Fig. 6 presents differences in KGE metrics between all three datasets. The maps in Fig. 6 are therefore obtained by subtracting the maps from Fig. 5, two at a time. The middle (ERA5) and right (ERA-I) columns present differences in hydrological modelling performance when using reanalyses compared to observations. A blue colour indicates that observations are superior for hydrological modelling, the reverse being true for red colours. This figure provides a clear view of the spatial patterns of hydrological modelling performance. Observations are clearly superior to reanalyses for the eastern half of the US. This corresponds to the zone with relatively large summer precipitation biases presented earlier in Fig. 3. Outside of this zone, both reanalyses perform similarly to observations, and especially so for ERA5. The left-hand side of Fig. 6 testifies to the uniform and significant improvement in hydrological modelling performance when using ERA5 compared to its predecessor ERA-I.

To gain a better understanding of the reasons behind these observations, hydrological modelling performance was analysed by looking at watershed size (Fig. 7), elevation (Fig. 8) and climate zone (Figs. 9 and 10). In those three cases, the results are only shown for the HMETS hydrological model, since the results for GR4J are similar, albeit with a small degradation in modelling performance, as shown in the preceding figures.

Since all three gridded datasets have different spatial resolutions, Fig. 7 looks at modelling performance for watersheds grouped under four different size classes. The patterns are consistent across all four size classes and similar to those of Fig. 4, with observations being best for all classes, followed by ERA5 and then ERA-I. However, it can be seen that hydrological modelling performance gets progressively better for larger watersheds for all three datasets. This is particularly clear for both reanalyses. While observations perform better at all scales, the gap with reanalysis gets smaller as catchment size increases. The interquartile range (defined by the solid rectangle of the boxplot) is roughly constant for observations, but consistently decreases for both reanalyses. Therefore, a larger proportion of smaller-size watersheds is challenging for hydrological modelling than for larger-size watersheds. Differences between ERA5 and ERA-I stay constant across all size classes.

Figure 8 presents the same data but as a function of watershed elevation, separated once again into four classes. Mean watershed elevation is mapped in Fig. 1. Figure 8 shows a strong dependence of hydrological modelling results on watershed elevation. Observations clearly perform better for the low-elevation (<500 m) watersheds, but differences rapidly shrink, with ERA5 actually performing as strongly and even better than observations for the last two elevation classes. It is relevant to stress that over 60 % of all watersheds are included in the first elevation class and that most of the eastern US watersheds are within the first two elevation classes. Results from Fig. 7 could therefore be influenced by watershed location in addition to elevation. It is also clear that ERA-Interim temperature gets progressively less competitive as the elevation rises, being significantly less efficient than ERA5 and the observations in the high-elevation groups.

The data were finally analysed by climate zone groupings. Figure 9 presents North America's climate classes from the Köppen–Geiger classification (Peel et al., 2007). It can be seen that North America displays four of the five main climate zones, with the exception of the equatorial climate. In total, 13 classes were kept for this analysis. Figure 10 presents hydrological modelling results for each of those 13 zones.

Results indicate that dataset performance and relative performance strongly depend on the climate zone. This is not surprising since performance was already shown to display spatial patterns. From Figs. 9 and 10, it is apparent that the ERA5 dataset is systematically better than ERA-Interim for all climate zones and that the observations are clearly superior to ERA5 for the Cfa and Dfa climate zones. Elsewhere, the differences are less pronounced. The Cfa and Dfa climate zones are the two main climate zones in the eastern US, which were shown to be problematic for the reanalysis datasets. Furthermore, ERA5 fares better than the observations in the northern parts of Canada and in the mountainous regions with climate zones Dfc and BSh, respectively. This observation will be discussed further, in Sect. 5.2. Figure 11 summarizes these results with the use of the Kruskal–Wallis statistical significance test to determine the best dataset for each climate zone. The Kruskal–Wallis hypothesis test is a non-parametric test to evaluate whether two samples originate from the same distribution. In Fig. 11, the green, yellow and red colours, respectively, indicate the best, second best and worst datasets for each climate zone. If two datasets share a colour for the same climate zone, the distribution of KGE values is considered to not be statistically different. Results indicate that there are no differences in hydrological modelling performance between ERA5 and observations over 9 of the 13 climate zones. For the other four regions (all in the eastern United States – Bsk, Cfa, Dfa, Dfb), using observations will result in a statistically significantly better hydrological modelling performance. ERA-I is the worst performing dataset over eight climate zones. In the remaining five zones, Bsh (3), Csa (53), Dsc (33), EF (3) and ET (15), all three datasets perform identically from a statistical viewpoint. These zones share in common the fewest watersheds and the most extreme climates (arid and polar).

In order to better explore the differences related to the watershed locations and properties, three catchments of different hydrological regimes were analysed in depth. Figure 12 presents the hydrological modelling KGE difference for HMETS between ERA5 and the observation dataset (first column) along with the mean monthly precipitation (second column), mean monthly temperature (third column) and mean annual hydrograph (fourth column). Results are presented for the Ouiska Chitto Creek near Oberlin, Louisiana, USA (first row), the Grande Rivière à la Baleine in Quebec, Canada (centre row) and the Cosumnes River at Michigan Bar, California, USA (bottom row). Table 1 shows summarized statistics for the three catchments.

The first row in Fig. 12 presents a catchment in the south-eastern United States, which is a region in which the reanalysis-driven hydrological models are unable to perform as well as the observation-driven models. ERA-Interim has a clear precipitation seasonality problem, being too dry except for the summer months, where there is a large overestimation of precipitation compared to the observations. This seasonality problem is mostly solved by ERA5, but a dry bias persists all year, as shown in Fig. 3. The temperatures between the three datasets are practically identical, which means that evapotranspiration should be relatively constant between the products. The lack of precipitation should therefore become apparent in the simulated hydrograph; however, the streamflow is higher for ERA5 than for the observations, when the opposite would normally be expected. It is important to note that the hydrological model can adapt its mass balance by adjusting the potential evapotranspiration scaling, which it has clearly done in this case. The difference in hydrological modelling then comes from the temporal distribution of precipitation, and it can be seen that the ERA5 winter precipitations are relatively lower in winter than for the rest of the year. The PET scaling therefore attempts to reduce evaporation for the entire year, but does not compensate enough to account for this difference in winter. Indeed, it can be seen that the observed hydrograph is underestimated by ERA5 and ERA-Interim for that period in the south-eastern United States.

The second catchment is located in northern Quebec, Canada, and as such is in a remote and sparsely gauged region. In this case, it can be seen that the ERA5-driven KGE metric is superior to that obtained using the observations. One key difference between the reanalysis and observed datasets is the precipitation, where ERA5 and ERA-Interim both show more precipitation than the observations. Again, the temperatures are practically identical, meaning that the potential evapotranspirations, although weak in that region, are very similar. The mean annual hydrograph is also very similar between ERA-Interim and the observations, but it can be seen that the ERA5 model overestimates streamflow in winter while matching the snowmelt peak flows more closely than the other datasets. The difference in KGE in this case comes from a better matching of peak flows, which counts more heavily towards the KGE than the low flows.

The third catchment, located in the west, is characterized by large precipitation systems in autumn and winter, with a months long dry spell in summer. ERA5 mostly corrected ERA-Interim's strong underestimation of precipitation for that catchment, as is the case for most western coast catchments as seen in Fig. 3. ERA5 temperatures are slightly cooler and are more in line with the observations. In terms of hydrological modelling, ERA-Interim underestimates the average streamflows year-round, while ERA5 slightly overestimates them in winter. As seen in Table 1, the ERA5 dataset managed to improve the KGE from 0.83 (ERA-Interim) to 0.87, as compared to the reference of 0.90 obtained with the observed data. The improvements in precipitation in ERA5 for this region thus seem to translate to improved hydrological modelling compared to using ERA-Interim, which confirms the findings of Fig. 6.

In this study, the observations are taken as the reference dataset and ERA5 is compared to both the observations and ERA-Interim. This allows validation of both the improvement in ERA5 with respect to ERA-Interim as well as evaluation of the possibility of using ERA5 reanalysis data as inputs to hydrological models to overcome potential deficiencies of observation networks, related to either quality and/or availability.

The evaluation of ERA5 temperature and precipitation variables compared to ERA-Interim and the observation datasets showed that ERA5 systematically reduced biases present in ERA-Interim for the temperature variables, whereas precipitation was generally also less biased, although to a lesser degree. There are remaining precipitation biases on the western coast of North America with ERA5, but from Fig. 2 it can bee seen that the scale of these biases is dependent on the season. In the south-eastern United States, ERA5 largely corrects biases that were present in the ERA-Interim dataset and led to relatively poor hydrological modelling in a few studies (e.g. Essou et al., 2016b). As for temperature, Fig. 2 shows that summer temperatures in ERA5 are mostly too high for the catchments west of the Rocky Mountains but are improved over the ERA-Interim data. There is also an interesting pattern of biases between the eastern and western coasts (Figs. 2 and 3), which could be partly explained by some processes not being accounted for in ERA5, notably the high-amplitude ridge trough wave patterns which have seen a recent increase allowing severe weather in both the east and west simultaneously (Singh et al., 2016; Raymond et al., 2017), although ERA5 did improve the representation of many processes since ERA-I (Hoffmann et al., 2019).

It is important to note that these perceived biases suppose that the observation data are perfect. In reality, at the catchment scale, one would expect that the observations would be far from perfect and contain errors due to location representativeness, precipitation undercatch, and missing data due to station malfunction or instrument replacement, for example. However, the observation data are the best estimates available, which makes them the de facto reference dataset. This means that although Figs. 2 and 3 show ERA5 and ERA-Interim as containing some important biases on western North America, it is possible that these biases are caused by biases in the station data relative to the catchment size. The reanalysis products also have the advantage of being driven by spatialized sources such as satellites, which can help in estimating precipitation and temperature data in regions where the weather station network is deficient or sparse.

One way to evaluate the quality of the observation and reanalysis data is to use hydrological models as integrators to compare simulated and observed streamflow, which can act as an independent validation variable. In an attempt to independently assess precipitation and temperature data for each dataset, all possible combinations of precipitation and temperature were fed to two hydrological models, which were then calibrated for each combination. This was to remove any bias caused by parameter sets calibrated on one single dataset, which would obviously be favoured in the resulting analysis. As was the case for the climatological variables, the observed streamflows act as the reference hydrometric data and are considered unbiased. Of course, in reality streamflow gauges contain various sources of errors (Di Baldassarre and Montanari, 2009), but for this study they are the best available estimates. This hypothesis could have a small effect on the conclusions of this study. For example, if a certain combination of precipitation and temperature datasets generates higher KGE calibration scores, it is assumed that the climate data are more likely to be correct than another dataset that returns lower KGE scores. This could be incorrect in some instances where the error actually comes from the streamflow data; however, on average over the 3138 catchments this effect should not influence the results.

The results in Fig. 4 showed that the hydrological models driven with the observed precipitation generally provide the most representative simulated hydrographs, with KGE values exceeding those of the ERA5-precipitation-driven hydrological models by 0.1 on average, which is a significant difference. ERA5 precipitation is also shown to be clearly better than ERA-Interim precipitation on average for the catchments in this study. Another interesting aspect is that in Fig. 4, replacing observed temperatures with ERA5 temperatures marginally improves the hydrological modelling skill. While not a significant difference, this attests to the quality of the ERA5 temperatures in general for hydrological modelling. Therefore, the differences observed in the hydrological modelling performance are almost entirely due to the precipitation data quality. The rest of this study will thus focus on the precipitation and hydrological modelling and forego further analysis of temperature data.

Also of note is that in general, ERA5-driven hydrological simulations are less skillful than those driven by observations. However, there are some catchments – mostly in the mountainous regions of the western United States and in northern Canada – where use of ERA5 leads to improved hydrological simulations. This is probably due to the difficulty in installing weather stations and obtaining representative observation data in those regions, but it shows that reanalysis data can be used as a replacement for observations for hydrological modelling in these regions, as previously reported by Essou et al., 2016b).

The more detailed spatial (Fig. 6) and climate zone (Figs. 10 and 11) analysis outlined the strong spatial dependence on dataset performance. Observations clearly outperformed ERA5 over the eastern half of the US, where a larger portion of the watersheds used in this study are located. To illustrate this point, Fig. 13 presents modelling performance over the eastern US (grouping climate zones Cfa, Dfa, and Dfb) against that of the other 10 climate zones.

Figure 13 paints a much different picture than Fig. 6 since it shows that hydrological modelling with ERA-5 precipitation and temperature is as good as observations everywhere in North America, with the exception of the eastern US. The disproportionate number of watersheds in this region may overemphasize the performance differential between ERA5 and observations as seen in Fig. 6. An interesting fact is that the eastern US is the North American region with by far the highest density of weather stations, as reported by Janis et al. (2002). Theoretically, this could explain why observation-based modelling performs better in this region. However, Fig. 13 shows that observation-based modelling performance is not different in the other regions, whereas reanalysis-based modelling clearly suffers over the eastern US. This was also noted in Essou et al. (2016b). It could mean that reanalyses face a harder challenge in the eastern US, further away from the Pacific Ocean control on atmospheric circulation. A large proportion of summer and autumn precipitation in these zones comes from convective storms. Eastern Canadian watersheds are well modelled using reanalyses, but the hydrological behaviour of most of those watersheds is dominated by the spring flood, which is largely controlled by temperature, which is very well reproduced by both reanalyses.

Alternatively, this could also mean that eastern US watersheds are in fact more difficult to hydrologically model and that differences are therefore directly linked to network density. Equal performance of ERA5 and observations elsewhere would therefore be the result of the improved process representation of ERA5 coupled with some degradation of observations due to the gridded interpolation process between more distant stations. As discussed below, a more precise investigation of modelling performance as a function of station density could shed light on this issue.

In this study, two hydrological models were selected to perform the hydrological evaluation of the reanalysis and observation datasets. While both models are conceptually similar, GR4J is simpler than HMETS (two routing processes instead of four, non-scalable PET, much simpler snow model, less than half the number of parameters, etc.). They were shown to perform generally well over all climate zones represented by the catchments used in this study, as can be seen in Fig. 4. Interestingly, both GR4J and HMETS return similar results for any given driving climate dataset. HMETS performs slightly better than GR4J almost everywhere, although that can be attributed to its more flexible model structure and parameterizations that can better adapt to various hydrological conditions.

Since the main objective of this study was to evaluate the ERA5 dataset for hydrological modelling, the interest is not to compare the hydrological model performances, but to compare the ERA5-driven simulations to the others for each model. In both cases, as can be seen in Figs. 4, 6 and 8, ERA5-driven hydrological models clearly outperform the ERA-Interim-driven models, which shows that the precipitation scheme in ERA5 is superior to that in ERA-Interim for hydrological modelling purposes. As stated in Sect. 5.2, temperature seems to play only a minor role in the differences in hydrological modelling.

Furthermore, the observation-driven hydrological models generally perform better than the ERA5-driven models, which confirms that station data should be prioritized when possible. The main caveat to this point is that when the observation station network is of poor quality or too sparse, then ERA5 can be used to fill the voids and get an acceptable hydrological response, as discussed in Sect. 5.2.

One of the major differences between ERA-Interim and ERA5 is the horizontal resolution, improving from 79 to 31 km. This finer resolution should allow for more precise estimations of precipitations and temperatures over smaller catchments that were not adequately represented by ERA-Interim. This logic should apply even though the hydrological models are lumped models. Larger catchments could also see some improvements, namely in a better estimation of the terrain elevation, but it is expected that the gain would not be as large as for smaller catchments.

In order to test this hypothesis, the improvements between ERA5 and ERA-Interim in hydrological modelling were sorted according to catchment size, as shown in Fig. 7. It is clear from Fig. 7 that the catchment size is not a good predictor of hydrological simulation improvement. While most catchments see improvements with ERA5 over ERA-Interim, the catchment size does not seem to affect the rate of improvement. This suggests that the improvements do not come from the higher spatial resolution, lending credence to the hypothesis that the enhancements are due to ERA5's improved physics and process representations.

A similar analysis was performed to evaluate the impact of catchment elevation on hydrological modelling skill. It can be seen from Fig. 8 that the elevation plays a significant role in the hydrological model's ability to estimate streamflow. For example, the median and interquartile ranges increase for all datasets as elevation increases. This could be caused by a more rapid hydrological response in higher-elevation and steeper catchments, compared to the slow runoff schemes often found in flat lowlands. The hydrological models being lumped models could contribute to this as large and flat catchments would be more affected by the location of rainfall events compared to steeper ones, especially in the timing of the hydrograph peaks. For the northern catchments, the peaks are caused by snowmelt which is much more uniform than rainfall events, which would minimize this effect.

Another, more probable reason for the reanalysis datasets being stronger in mountainous regions is simply because there are fewer weather stations set up in those areas due to difficulties in accessing and maintaining them. The density of weather stations in the eastern part of the US is typically at least twice as large as for the western part (Janis et al., 2002). In such cases, a reanalysis would provide information that is not conveyed by station data, making it a de facto best estimation of precipitation. In essence, the ERA5 data are not yet as accurate as observations; however, they are able to perform very well in their absence.

Finally, in all the analysed scenarios in this study, ERA5 has always been at least as good as ERA-Interim in terms of hydrological performance. The same is true for the precipitations and temperatures at the catchment scale. From all the results in this study, there does not seem to be any reason or indication that ERA-Interim should continue to be used for hydrological modelling applications, at least in North America. This is not to say that ERA5 is perfect, but it should become the reference for the time being.

As is the case with any large-scale comparison studies, some methodological limitations may potentially impact conclusions drawn from the presented results. In terms of hydrological modelling, this study only uses two lumped conceptual models and one flow criterion (KGE). Both models are lumped, which limits the assessment of the horizontal resolution component of the three datasets. This aspect was however indirectly assessed by looking at the impact of watershed size. Both hydrological models are conceptually similar, but HMETS is more flexible and has more hydrological processes (and parameters). Accordingly, this study was able to look at the impact of parametric space flexibility in dealing with various dataset biases, but not at other issues such as the impact of physically based processes and distributed inputs. A study looking at the latter points would require more complex hydrological models, but at the expense of having to look at far fewer watersheds.

The single streamflow criteria and objective function (KGE), like its Nash–Sutcliffe relative, is weighted towards higher-flow events. Other objective functions would return different results; however, the fact that ERA5 climate data are generally improved in all areas is an indicator that other metrics could potentially see improved results as well, although no test has been performed to that effect in this study. There are several other streamflow criteria which could shed light on differences between datasets, such as extremes. In particular, high-flow extremes have the potential to outline improvements in ERA5 compared to its predecessor ERA-I because of improved resolution and processes. Low flows may also be of interest, although they are typically less well modelled by conceptual hydrological models and are more strongly dependent on temperature, which is very comparable across all three datasets. Finally, there are now several potential other precipitation datasets that could have been included in the comparison (see for example Beck et al., 2017a). However, the goal of this work was a first evaluation of the 1979–2019 ERA5 dataset, because of the potential linked to its spatial and temporal resolutions.

One of the main reasons for the interest in the ERA5 reanalysis resides in its hourly temporal resolution. Therefore, the obvious next step is to investigate sub-daily components, and particularly for precipitation. Sub-daily precipitation is key to investigating the hydrological response of smaller watersheds. However, sub-daily studies raise another set of challenges, notably the absence of a robust baseline hourly meteorological dataset. MSWEP (Beck et al., 2017b) is the best potential candidate at the sub-daily timescale (3-hourly), but the reliability of its sub-daily component is largely unknown. Reliance on hourly weather station data will therefore be required, meaning additional problems, including having to deal with missing data.

The differences noted in the eastern USA raised the question of the potential impact of the density of the station network on the absolute and relative performance of the various datasets. This could be better studied by assigning a network density index to each watershed. This could ultimately lead to a better understanding of the role of station density and provide guidance on network improvements or rationalization. It could also be envisioned to extend this work to underdeveloped countries where there is a lower number of observational gauges, where a good quality reanalysis might allow for improved hydrological simulations and better understanding of the regional weather characteristics.

The hydrological performance of ERA5 opens specific avenues of research for streamflow forecasting using ECMWF forecasts. Calibrating hydrological models with ERA5 data could potentially reduce streamflow forecast biases since the reanalysis and forecasts essentially originate from the same model.