General circulation models (GCMs) and Earth system models (ESMs) are the primary tools used to simulate the response of the global climate system to increases in greenhouse gas concentrations and to generate future climate projections. GCMs are complex mathematical representations of the physical and dynamical processes governing atmospheric and oceanic circulations as well as the interactions with the land surface. In order to reduce the computational burden, which can be considerable, GCMs represent the Earth with a grid that has a relatively coarse spatial resolution . Consequently, GCM projections cannot be used directly for fine-scale climate impact studies. Statistical and empirical or dynamical downscaling techniques have, thus, been commonly used to address this scale mismatch. In addition, climate model outputs are always biased, and the extent of these biases can be evaluated through a comparison against observations over a common reference period. A bias correction procedure is therefore generally performed in addition to the downscaling step, and biases are assumed to be invariant in time when the correction is applied to future climate projections . Although a two-step downscaling bias correction approach is preferable in most cases, a single instance of bias correction is sometimes used to account for both scale mismatch and GCM biases. While this may be acceptable when the scale difference is small (e.g., when using catchment-averaged values), recent studies have shown that bias correction has limited downscaling skills .

Statistical downscaling and bias correction approaches primarily rely on hydrometeorological observations over a historical reference period. It is, therefore, primordially important that the observed reference dataset represents the true climate state as closely as possible. For this task, ground stations remain the standard and most accurate and trusted source of weather data . However, the spatial distribution of these stations varies widely across the globe, and coverage is often sparse and even deficient in many parts of the world outside of Europe and the USA. Even in well-covered regions, gauge data are subject to many problems, such as missing data, precipitation undercatch and inhomogeneities related to a variety of issues such as equipment change, station relocation and land surface modifications near each station .

In recent decades, extensive efforts have been devoted to the development and improvement of gridded global and quasi-global climate datasets to overcome the limitations of gauge stations. These datasets provide meteorological record time series with continuous spatiotemporal coverage and, typically, no missing data. However, various error sources are inherent in these datasets, thus also bringing uncertainty to the data . Thus, choosing an appropriate reference dataset for climate change impact studies is an important concern, and this is especially so in regions with sparse ground station coverage.

According to , “For estimates based on downscaling of general circulation model (GCM) outputs, different levels of uncertainty are related to (1) GCM uncertainty or intermodel variability, (2) scenario uncertainty or inter-scenario variability, (3) different realizations of a given GCM due to parameter uncertainty (inter-model variability) and (4) uncertainty due to downscaling methods.” In most climate change impact studies, it is generally assumed that GCMs are the major source of uncertainty . compared the effect of different sources of uncertainty using the initial condition ensembles of different GCMs, greenhouse gases emission scenarios (GHGESs) and regional circulation models (RCMs) on changes in seasonal precipitation and temperature in the United Kingdom. The results indicated that the largest uncertainty comes from the GCM choice. used 10 equally weighted climate projections derived from a combination of five GCMs, two GHGES and a single downscaling method for downscaling to investigate the uncertainty envelope of future hydrologic variables. Their results showed that the uncertainty related to the GCM choice is dominant. These results have also been confirmed by several studies . Other studies have assessed other sources of uncertainty such as GHGESs , the downscaling method and hydrological modeling . Recent studies have also looked at the uncertainty related to the choice of the impact model . From these studies, a more complex picture emerges, in which the main source of uncertainty may vary, depending on geographical location and metric under study. Dataset uncertainty has been assessed in numerous studies, either by direct intercomparison between datasets or by using hydrological modeling . However, to the best of our knowledge, the uncertainty of gridded datasets has not been evaluated against other sources of uncertainties when performing climate change impact studies. The objective of this study is, therefore, to assess the impact of the choice of a given reference dataset on the global uncertainty chain of climate change impact studies. Since this is of particular concern to regions with sparse weather station coverage, this study is conducted over Africa.

Figure 2 presents the methodological framework for this study. A large sample hydrological climate change impact study is performed over 1145 African catchments. It uses the standard top-down approach in a modeling chain, which consists of 10 GCMs, two hydrological models, two temperature and nine precipitation datasets for a total of 360 possible combinations. A single GHGES (RCP8.5), a single climate projection for each GCM and a single downscaling method (see below) are used, since the focus of this work is not on conducting a complete uncertainty chain study. The uncertainty related to the reference dataset will, therefore, be compared to that of the climate model ensemble and against that of both hydrological models. These two sources are generally considered to be the most important in climate change impact studies e.g.,. For each catchment, 360 30 year streamflow time series are generated for both the reference (1983–2012) and future (2071–2100) time periods. A total of six streamflow metrics are computed for each of these time series. An n dimensional analysis of variance is performed to partition the uncertainty linked to the four selected groups of components of the uncertainty modeling chain, namely precipitation and temperature datasets and GCMs and hydrological models.

Both hydrological models were calibrated on all catchments for all 18 combinations of reference datasets (two temperature datasets × nine precipitation datasets), for a total of 41 220 independent hydrological model calibrations. Combining different – and somewhat independent – data sources for temperature and precipitation raises potential issues about the mass and energy balance. Most of the products used in this work originate from a gridding process that is independently done for precipitation and temperature, which, therefore, do not take temporal correlations between both variables into account. Most precipitation products are also developed independently of temperature. Reanalyses are the most consistent dataset with respect to energy budget and water balance. However, even though the weather model of the reanalysis is entirely physically coherent, the data assimilation does not preserve this physical coherency and, therefore, reanalysis does not conserve water balance. The combination of precipitation and temperature datasets is therefore unlikely to be problematic. More details about the calibration process are described later in Sect. 3.1.3.

The watershed boundaries for the African continent were extracted from the HydroSHEDS database. Streamflow records from the GRDC database were used to calibrate the hydrological models and to evaluate the hydrological modeling performance. In this study, 350 stations were chosen from the GRDC database based on three criteria. First, stations should have data for the 1983–2012 study period. Second, stations that have fewer than 5 consecutive years of data during this period were excluded. Finally, all the stations should be compatible with the selected HydroSHEDS catchments. In order to include additional catchments to allow for a better spatial coverage over the African continent, an additional 795 catchments (the remaining catchments from the Lev05 layer of Fig. 1) were selected and an additional regionalization step was performed to generate streamflows at these 795 catchments. The climatological data from nine precipitation and two temperature datasets were then extracted for each of these 1145 catchments. The main methodological steps are described in Fig. 2.

This section outlines the main findings of the work. Figure 3 presents the calibration results for both hydrological models using all possible combinations of the nine precipitation and two temperature datasets. Each box plot consists of 350 KGE values corresponding to the calibration result for each of the 350 selected gauged catchments. Each box extends from the 25th quantile to the 75th quantile, with the median displayed as the red line within that range. The top and bottom whiskers (where shown) represent highest and lowest values. Red crosses are considered statistical outliers.

Results show that both hydrological models perform well, but that there are important differences between datasets. HMETS performs better than GR4J, with respective overall mean KGEs of 0.58 and 0.41. All the precipitation and temperature datasets result in acceptable median KGE simulations .

Based on the models' calibration performance, both temperature datasets perform similarly across all combinations, with ERA5 generally slightly outperforming CPC. Figure 3 clearly shows that most of the variability seen originates from the precipitation datasets; four precipitation datasets are ahead of the field. They are, in order of performance, the merged product MSWEP, followed by the two satellite datasets, CHIRPS and PERSIANN, and the ERA5 reanalysis dataset. The gauge-based precipitation datasets (e.g., GPCC and CPC) and the ERA-I reanalysis follow with a similar performance. Finally, the CFSR and JRA55 reanalysis are the worst-performing products for hydrological model calibration.

Table 3 presents the main results of the ANOVA for the 2071–2100 period for the gauged catchments. It shows the relative variance for all main effect and first-order interactions of the four components of uncertainty under study and for six streamflow metrics. The variance originating from second- and third-order interactions are summed up and presented in the last row. Results show that most of the variance consistently comes from five sources for all six streamflow metrics. They are the precipitation (P) datasets, GCMs, hydrological models (HMs), interactions between precipitation datasets and GCMs (P-GCM), as well as interactions between precipitation datasets and hydrological models (P-HMs).

Table 3 indicates that both the precipitation datasets and GCMs are the main contributors to variance, including through interactions (P-GCM). The hydrology models also generate some uncertainty, particularly through interaction with the precipitation datasets. All metrics exhibit a similar pattern, with the exception of the low-flow metric (QQ5), where precipitation, hydrological models and their interaction components (P-HMs) are dominant and for which GCM uncertainty is minimal. In almost all cases, the five highlighted components represent approximately 85 % of the total variance. The average amount of variance introduced by both temperature datasets is less than 0.25 % for all six different streamflow metrics.

To show cross-catchment variability, Fig. 4 shows box plots of the relative variance attribution results for the five main contributors to variance, as identified in Table 3, and for the same six streamflow metrics. The results are also decomposed into three parts, i.e., all 1145 catchments (A) and the 350 gauged (G) and 795 ungauged (U) catchments, in order to ensure that the regionalization process does not introduce undesirable effects on the results.

Figure 4 shows that the response of the gauged and ungauged catchments is very similar across all variance components and streamflow metrics, and that no major variance artifact is introduced by the regionalization step. Consequently, all further results will only be shown for all 1145 catchments, with no differentiation made between the gauged and ungauged ones.

The results show that there is considerable cross-catchment variability, as shown by the extent of the box plots, with GCM and P-GCM interactions being the most important and most variable contributors to variance. As was shown in Table 3, the low-flow metric displays a pattern that is very different to the other five metrics, with HM being important and GCM being the lowest. Also, the HM and P-HM show significant contributions to the uncertainty in the summer Q metric. There is a relatively large difference between the two metrics representing high flows (QQ95 and QX1). While GCM dominates the former, a much larger part of the uncertainty is transferred to the precipitation dataset (P and P-GCM) for the latter.

In order to study the impact of spatial variability, Fig. 5 presents the spatial distribution of the relative variance attribution for the five main contributors to variance of Table 3 and all six streamflow metrics. Mean Q, winter Q, QX1 and QQ95 display somewhat similar spatial patterns. Summer Q and QQ5 metrics display somewhat similar spatial patterns. The largest precipitation uncertainty (P and P-GCM interactions) is found in the northern parts of sub-Saharan Africa, between 0 and 30∘ N. GCM uncertainty appears to be larger all around the coastlines of Africa. HM uncertainty is strongest for QQ5, but spatial patterns are fairly consistent across all six streamflow metrics. GCM uncertainty is strongly different for both summer Q and winter Q, likely because of the monsoon pattern. Above 20∘ N, there is generally less than 100 mm of total annual precipitation, and some level of care should therefore be taken when analyzing results in relative contribution to variance. The relative contribution to variance is not related to absolute mean streamflow values, and therefore, the color scale is the same for major rivers and smaller intermittent streams. Many of the catchments above 20∘ N run dry for a large part of the year.

In other words, a variance analysis of a metric with very little absolute variance could be misleading. Consequently, Fig. 6 displays the standard deviation of the 360 streamflow values computed for each streamflow metric and for each watershed. Therefore, Fig. 6 does not represent the variance contribution of any given component of the hydroclimatic chain but represents the total variance of all components combined. A low value indicates that a streamflow metric shows little variability across its 360 values. This would be expected, for example, for catchments with a high aridity index resulting in very transient flow. The streamflow value for each metric is normalized per unit area to allow for a comparison of large and small watersheds in the same figure. Not surprisingly, the results demonstrate a larger variance along the equatorial band where precipitation is largest. This pattern is particularly clear for the QQ95 high-flow metric. The catchment database is, however, large enough to show some catchments which exhibit a large variance, even in arid regions above 20∘ N and below 20∘ S.

Since some precipitation datasets are clearly better than others, based on the hydrological model calibration results, it may not be entirely fair to compare precipitation uncertainty to GCM uncertainty. To investigate this further, the uncertainty contribution obtained when using all nine precipitation datasets is compared to that of three sub-ensembles, as presented in Table 4. While ensemble 4 is clearly composed of the best-performing datasets for model calibration, the main goal here is to investigate the impact of dataset selection and not the definition of a credibility ensemble, as will be further discussed later.

Figure 7 presents the box plots of percentages of variance for each catchment for the five main contributors to variance for all four precipitation dataset ensembles of Table 4. Unsurprisingly, it shows that reducing the size of the precipitation ensemble results in a consistent decrease in the variance attributed to precipitation. Most of this reduction in variance comes from the P-GCM interaction term, although there is also a noticeable decrease in the main effect P component. The lost precipitation variance is transferred mostly to GCMs and, to a lesser extent, to hydrological modeling. The exception is the low-flow QQ5, where most of the variance is transferred to HM. Most of the drop observed is obtained by dropping the five worst precipitation datasets (ensemble 4), as no significant difference is observed between precipitation ensembles 3 and 4. Even in a reduced ensemble, precipitation datasets still provide between 10 % and 20 % of median variance and more than 30 % for the low-flow metric (QQ5) when taking into account the main effect and first-order interaction term.

Figure 8 presents the spatial distribution of the relative variance attribution for each of the six streamflow metrics after including only the four best overall precipitation datasets (ensemble 4 in Table 4). This is the same as Fig. 5 but with a reduced precipitation ensemble. Results outline that GCM uncertainty is the dominant source of uncertainty when using the reduced precipitation ensemble, with the exception of the low-flow metric for which hydrological model uncertainty is dominant. There are, however, significant interactions between GCM and precipitation for all metrics, especially in the northern half of the continent. Otherwise, the observed spatial patterns are similar to the ones presented in Fig. 5.

Defining a reference climate dataset is an important but difficult task. A reference climate dataset is used as a benchmark for monitoring environmental changes and correcting climate model biases of future climate projections to assess future impacts of a changing climate. Data from weather stations are still mostly considered to be the most accurate representation of the current climate, despite suffering from several important issues, such as precipitation undercatch and inhomogeneities . To allow for regular data coverage and remove missing data, it is now common practice to interpolate station data onto a regular grid. Such gridded datasets greatly simplify the processing of meteorological data for environmental studies at the regional, continental and global scales. However, even in regions with a good weather station coverage, gridded datasets using the same underlying data differ due to the different interpolation methods and typically see an increase in the number of wet days and a decrease in the frequency of extreme events . In regions with scarce weather station coverage (such as Africa), interpolation becomes extrapolation and is therefore potentially highly unreliable. In such cases, environmental studies have had to rely on additional sources of data, such as satellite and atmospheric reanalysis for environmental studies. Several intercomparison studies have been done (e.g., ), including over Africa . These studies outline a complex picture in which performance depends on scale, climate and data source and for which no dataset consistently outperforms all of the others. Because of this, in data-sparse regions such as Africa, there is not only no commonly agreed upon reference dataset but there is also no agreement on the optimal source of climate data (e.g., satellite vs. reanalysis), and different environmental studies have used completely different datasets. This is particularly problematic for climate change impact studies since there is no knowledge of how dataset uncertainty may propagate in the typical hydroclimatic modeling chain. The results presented in this study attempt to answer this question by comparing dataset uncertainty to other sources of uncertainty, such as that derived from GCMs.

Results show that most of the dataset uncertainty originates from precipitation. Temperature displays much smaller spatial and temporal variability than precipitation and can, therefore, be a lot more reliably interpolated by using the adiabatic lapse rate to account for elevation and terrain orientation in mountain areas. Precipitation interpolation is a much more challenging problem, which explains why most dataset intercomparison work has focused on this variable. Based on KGE performance over a common reference period, all nine precipitation datasets performed adequately in terms of hydrological modeling performance, but some clearly performed much better than others. This is in agreement with the results of and . The uncertainty contribution of datasets to future streamflow uncertainty was first evaluated using all nine precipitation datasets in conjunction with two temperature datasets, a sample of 10 GCMs and two hydrological models for a total of 360 possible element combinations. While this is a relatively large sample, not all sources of uncertainty were accounted for. In particular, GHGESs, downscaling and bias correction were not included in the analysis. In comparison, the North American Climate Change and Hydroclimatology Dataset (NAC2H) database offers 16 000 combinations, allowing the examination of future streamflow uncertainty. In this regard, the relative variance contribution of the climate dataset is best examined in comparison to that of GCMs, the most studied source of climate change impact uncertainty. Results outline the important and, in some cases, dominant contribution of the precipitation dataset to the overall uncertainty of future streamflows. For all six streamflow metrics presented here, the precipitation dataset uncertainty was comparable and sometimes larger than that of the GCMs.

Uncertainty contribution was then studied by retaining subsets of precipitation datasets and eliminating the least-performing ones with respect to the chosen KGE metric. This follows the concept of a credibility ensemble based on carefully selecting the best or most robust components of the hydroclimatic modeling chain in order to obtain the most credible uncertainty range (). Results demonstrate a large decrease in contribution to uncertainty for five of six streamflow metrics. The precipitation dataset remained the largest contributor to uncertainty for the low-flow metric and still accounted for 10 % to 20 % of the total variance for the other metrics. Much of the decrease in uncertainty was obtained by dropping the worst-performing datasets rather than keeping the best-performing ones.

The results presented here indicate that hydrological model uncertainty is relatively small, with the exception of the low-flow metric. These results should be taken with caution because only two hydrological models were used and also because they both share the same potential evapotranspiration (PET) formula. For climate change impact studies, the climate sensitivity of PET is now thought to be an important source of uncertainty for impact studies , and the importance of hydrological model uncertainty has been outlined in many studies . A better understanding of how hydrological model components affect uncertainty would therefore be very valuable for climate change impact studies e.g.,. Taking the above into consideration, it is therefore likely that the contribution of hydrological models is underestimated here. The number of components in a variance attribution study is an important issue. However, the contribution to variance is related to how dissimilar the ensemble members are and not strictly to their numbers. As such, the ensemble with the fewest members can still provide the largest contribution to variance. There is, therefore, no need for all ensembles to have the same number of members but rather to have enough credible members to cover the uncertainty. Despite having only two temperature datasets here, adding more temperature datasets is unlikely to change the results considering how little uncertainty is present in the two datasets when compared to other sources. Temperature is the easiest variable to measure and to extrapolate, especially when compared to precipitation. It is, therefore, expected that precipitation uncertainty would normally dwarf the contribution of temperature. Based on previously published work, 10 GCMs is very likely more than enough to frame the uncertainty contribution from this source e.g.,.

The selection of the best-performing precipitation dataset was evaluated over a reference period using the single metric of the KGE criterion. This criterion is considered to be a good metric as it weighs bias, correlation and root mean square error (RMSE) between simulation and observations, all rightfully considered to be important attributes of a good hydrological simulation. There are, however, many other metrics that could have been chosen to perform this comparison, some of which might be even more important for specific applications such as floods. For example, the JRA55 and CFSR reanalyses were at the bottom of the list of the best-performing datasets presented here. However, in other studies, JRA55 was shown to provide the best reanalysis , while CFSR was successfully used for precipitation modeling . Clearly, the results presented in this paper should only be used as intended (i.e., to study uncertainty related to the choice of a reference climate dataset) and not as a judgment of the absolute performance of each dataset. As mentioned earlier, it is important to keep in mind that all of the datasets used in this paper generate adequate streamflow simulations.

It is recommended that reference dataset uncertainty be included in climate change impact studies and especially so in regions with a sparse network of weather stations. We believe that climate dataset uncertainty can be minimized for most streamflow metrics using a careful validation and selection of the best-performing ones. A dataset ensemble should nonetheless be retained to assess the sensitivity of the impact study to the choice of a reference dataset. As is the case for most other elements of the hydroclimatic modeling chain of future climate change impacts, there is no free lunch in the sense that there is no single recipe which will be applicable in all cases. Climate dataset performance is spatially dependent, as shown here and in other studies, and will depend on the criteria used to assess said performance. In addition, the relative uncertainty contribution also depends on the catchment location and streamflow metric under study. The importance of first-order interactions in variance analysis, and especially of interactions between precipitation datasets with GCMs and with the hydrology models, testify to the complex nature of the propagation of uncertainties in the hydroclimatic modeling chain. The use of an appropriate credibility climate dataset ensemble is therefore more than likely to be catchment related and metric dependent, and some minimum level of upstream validation would be needed to select the best components.

Some level of guidance for impact modelers can nonetheless be offered from the results of this work. Precipitation is the key driver of dataset uncertainty and should, therefore, be evaluated in climate change studies alongside the more traditional sources of uncertainty. In cases where it is not possible to select multiple precipitation datasets, the results presented in Fig. 3 and in indicate that the MSWEP merged product dataset is the best-performing one, with CHIRPS and ERA5 being the next best. The gauge-only-based products were clearly not the best-performing ones over Africa, which is in contrast to a similar study performed over North America . This performance ranking is, however, only based on the KGE calibration metric. While the KGE is a good overall performance metric, it is possible that using a different performance metric might affect this ranking. Streamflow data also come with many potential quality issues that must be taken into consideration e.g.,. However, in the overwhelming majority of cases, there are no competing streamflow datasets from which to study uncertainty, but flawed streamflow records will impact hydrological model calibration and performance and may, therefore, indirectly contribute to hydrological model uncertainty.

The main objective of this study was to assess the uncertainty related to the choice of a reference dataset against that of other sources of uncertainty in climate change impact studies. This was achieved by performing a large-sample hydrological climate change impact study over 1145 African catchments. The study used nine precipitation and two temperature datasets, along with 10 GCMs and two hydrological models, for a total of 360 possible combinations. Temperature data-set-related uncertainty was minimal, with a median relative contribution to uncertainty of less than 0.25 % for all six presented streamflow metrics. On the other hand, the nine precipitation dataset ensembles generated a future uncertainty equal to or larger than that related to GCMs. Using a reduced ensemble of the best-performing precipitation datasets systematically reduced the precipitation dataset uncertainty but still accounted for 10 % to 20 % of the total variance for five of the six streamflow metrics and still remained the main source of uncertainty for the low-flow metric. The main conclusion of this study is that the choice of a climate reference dataset can induce significant uncertainty in climate change impact studies, at least in regions with sparse weather station coverage.