We evaluated the performance of 26 gridded (sub-)daily P datasets (Tables  and ). All datasets are either fully or near global, with the exception of WRF, which is limited to the CONUS. The datasets are classified as either uncorrected, which implies that temporal variations depend entirely on satellite and/or (re)analysis data, or corrected, which implies that temporal variations depend to some degree on gauge observations. We included seven datasets exclusively based on satellite data (CMORPH V1.0, GSMaP-Std V6, IMERGHHE V05, PERSIANN, PERSIANN-CCS, SM2RAIN-CCI V2, and TMPA-3B42RT V7), six fully based on (re)analyses (ERA-Interim, ERA5-HRES, ERA5-EDA, GDAS-Anl, JRA-55, and NCEP-CFSR, although ERA5 assimilates radar and gauge data over the CONUS), one incorporating both satellite and (re)analysis data (CHIRP V2.0), and one based on a regional convection-permitting climate model (WRF).

Among the gauge-based P datasets, six combined gauge and satellite data (CMORPH-CRT V1.0, GPCP-1DD V1.2, GSMaP-Std Gauge V7, IMERGDF V05, PERSIANN-CDR V1R1, and TMPA-3B42 V7), one combined gauge and reanalysis data (WFDEI-GPCC), three combined gauge, satellite, and (re)analysis data (CHIRPS V2.0, MERRA-2, and MSWEP V2.2), while one was fully based on gauge observations (CPC Unified V1.0/RT). For transparency and reproducibility, we report dataset version numbers throughout the study for the datasets for which this information was provided. For the P datasets with a sub-daily temporal resolution, we calculated daily accumulations for 00:00–23:59 UTC. P datasets with spatial resolutions <0.1∘ were resampled to 0.1∘ using bilinear averaging, whereas those with spatial resolutions >0.1∘ were resampled to 0.1∘ using bilinear interpolation.

As a reference, we used the NCEP Stage-IV dataset, which has a 4 km spatial and hourly temporal resolution and covers the period 2002 until the present, and merges data from 140 radars and ∼5500 gauges over the CONUS . Stage-IV provides highly accurate P estimates and has therefore been widely used as a reference for the evaluation of P datasets (e.g., ). Daily Stage-IV data are available, but they represent an accumulation period that is incompatible with the datasets we are evaluating (12:00–11:59 UTC instead of 00:00–23:59 UTC). We therefore calculated daily accumulations for 00:00–23:59 UTC from 6-hourly Stage-IV accumulations. The Stage-IV dataset was reprojected from its native 4 km polar stereographic projection to a regular geographic 0.1∘ grid using bilinear averaging.

The Stage-IV dataset is a mosaic of regional analyses produced by 12 CONUS River Forecast Centers (RFCs) and is thus subject to the gauge correction and quality control performed at each individual RFC . To reduce systematic biases, the Stage-IV dataset was rescaled such that its long-term mean matches that of the PRISM dataset for the evaluation period (2008–2017). To this end, the PRISM dataset was upscaled from ∼800 m to 0.1∘ using bilinear averaging. The PRISM dataset has been derived from gauge observations using a sophisticated interpolation approach that accounts for topography. It is generally considered the most accurate monthly P dataset available for the US and has been used as a reference in numerous studies (e.g., ). However, the dataset has not been corrected for wind-induced gauge undercatch and thus may underestimate P to some degree .

The evaluation was performed at a daily temporal and 0.1∘ spatial resolution by calculating, for each grid cell, KGE scores from daily time series for the 10-year period from 2008 to 2017. KGE is an objective performance metric combining correlation, bias, and variability. It was introduced in and modified in and is defined as follows: KGE=1-r-12+β-12+γ-12, where the correlation component r is represented by (Pearson's) correlation coefficient, the bias component β by the ratio of estimated and observed means, and the variability component γ by the ratio of the estimated and observed coefficients of variation: β=μsμoandγ=σs/μsσo/μo, where μ and σ are the distribution mean and standard deviation, respectively, and the subscripts s and o indicate estimate and reference, respectively. KGE, r, β, and γ values all have their optimum at unity.

Figure  presents box-and-whisker plots of KGE scores for the 26 P datasets. The mean median KGE score over all datasets is 0.54. The mean median scores for the correlation, bias, and variability components of the KGE, expressed as |r-1|, |β-1|, and |γ-1|, are 0.34, 0.18, and 0.16, respectively (see Eq. ). The datasets thus performed considerably worse in terms of correlation, which makes sense given that long-term climatological P statistics are easier to estimate than day-to-day P dynamics. Due to the squaring of the three components in the KGE equation (see Eq. ), the correlation values exert the dominant influence on the final KGE scores. Indeed, the performance ranking in terms of KGE corresponds well to the performance ranking in terms of correlation (Fig. ). These results suggest that in order to get an improved KGE score, the most important component score to improve is the correlation. This in turn suggests that, for existing daily P datasets, improvements to the timing of P events at the daily scale (dominating the correlation scores) are more valuable than improvements to P totals (dominating bias scores) or the intensity distribution (dominating variability scores).

Among the uncorrected P datasets, the (re)analyses performed better overall than the satellite-based datasets (Figs.  and ). The best performance was obtained by ECMWF's fourth-generation reanalysis ERA5-HRES (median KGE of 0.63), with NASA's most recent satellite-based dataset IMERGHHE V05 and the ensemble average ERA5-EDA coming a close equal second (median KGE of 0.62). These results underscore the substantial advances in earth system modeling and satellite-based P estimation over the last decade. The third-generation, coarser-resolution reanalyses (ERA-Interim, JRA-55, and NCEP-CFSR) performed slightly worse overall (median KGE of 0.55, 0.52, and 0.52, respectively). ERA-Interim performed slightly better than other third-generation reanalyses, consistent with earlier studies focusing on P and other atmospheric variables . All (re)analyses, including the new ERA5-HRES, underestimated the variability (Figs.  and S3 in the Supplement), reflecting the tendency of (re)analyses to overestimate P frequency . The additional variability underestimation by ERA5-EDA compared to ERA5-HRES probably reflects the variance loss induced by the averaging.

Among the uncorrected satellite-based P datasets, the new IMERGHHE V05 performed best overall by a substantial margin (median KGE of 0.62; Figs.  and ), reflecting the quality of the new IMERG P retrieval algorithm . The other passive-microwave-based datasets (CMORPH V1.0, GSMaP-Std V6, and TMPA-3B42RT V7) obtained median KGE scores ranging from 0.44 to 0.52. CHIRP V2.0, which combines infrared- and reanalysis-based estimates, performed similarly to some of the passive-microwave datasets (median KGE of 0.47). The datasets exclusively based on infrared data (PERSIANN and PERSIANN-CCS) performed markedly worse (median KGE of 0.34 and 0.32, respectively), consistent with previous P dataset evaluations (e.g., ). This has been attributed to the indirect nature of the relationship between cloud-top temperatures and surface rainfall . The infrared-based datasets generally exhibited a much larger spatial variability in performance for all four metrics (Figs.  and S1–S3).

The (uncorrected) satellite soil moisture-based SM2RAIN-CCI V2 dataset performed comparatively poorly (median KGE of 0.28; Figs.  and ). The dataset strongly underestimated the variability (Fig. S3), due to the noisiness of satellite soil moisture retrievals and the inability of satellite soil moisture-based algorithms to detect rainfall exceeding the soil water storage capacity . At high latitudes and elevations, the presence of snow and frozen soils may have hampered performance , while in arid regions, irrigation may have been misinterpreted as rainfall . In addition, approximately 25 % (in the eastern CONUS) to 50 % (over the mountainous west) of the daily rainfall values were based on temporal interpolation, to fill gaps in the satellite soil moisture data . Despite these limitations, the SM2RAIN datasets may provide new possibilities for evaluation and correction of other P datasets, since they constitute a fully independent, alternative source of rainfall data.

All uncorrected P datasets exhibited lower overall performance in the western CONUS (Figs. , , and S1–S3), in line with previous studies (e.g., ). This is attributable to the more complex topography and greater spatio-temporal heterogeneity of P in the west , which affects the quality of both the evaluated datasets and the reference . With the exception of CHIRP V2.0 (which has been corrected for systematic biases using gauge observations; ) and WRF (the high-resolution climate simulation; ), the (uncorrected) datasets exhibited large P biases over the mountainous west (Fig. S2), which is in agreement with earlier studies using other reference datasets and reflects the difficulty of retrieving and simulating orographic P . We initially expected bias values to be higher than unity since PRISM, the dataset used to correct systematic biases in Stage-IV (see Sect. ), lacks explicit gauge undercatch corrections , but this did not appear to be the case (Figs.  and S2).

Among the gauge-based P datasets, the best overall performance was obtained by MSWEP V2.2 (median KGE of 0.81), followed at some distance by IMERGDF V05 (median KGE of 0.67) and MERRA-2 (median KGE of 0.66; Figs.  and ). IMERGDF V05 exhibited a small negative bias, while MERRA-2 slightly underestimated the variability. The good performance obtained by MSWEP V2.2 underscores the importance of incorporating daily gauge data and accounting for reporting times . While CMORPH-CRT V1.0, CPC Unified V1.0/RT, GSMaP-Std Gauge V7, and MERRA-2 also incorporate daily gauge data, they did not account for reporting times, resulting in temporal mismatches and hence lower KGE scores (Fig. ). Reporting times in the CONUS range from midnight -12 to +9 h UTC for the stations in the comprehensive GHCN-D gauge database (; Fig. 2c in ), suggesting that up to half of the daily P accumulations may be assigned to the wrong day. In addition, CMORPH-CRT V1.0, GSMaP-Std Gauge V7, and MERRA-2 applied daily gauge corrections using CPC Unified , which has a relatively coarse 0.5∘ resolution, whereas MSWEP V2.2 applied corrections at 0.1∘ resolution based on the five nearest gauges for each grid cell . The good performance of IMERGDF V05 is somewhat surprising, given the use of monthly rather than daily gauge data, and attests to the quality of the IMERG P retrieval algorithm .

Similar to the uncorrected datasets, the corrected estimates consistently performed worse in the west (Figs. , , and S1–S3), due not only to the greater spatio-temporal heterogeneity in P , but also the lower gauge network density . It should be kept in mind that the performance ranking may differ across the globe depending on the amount of gauge data ingested and the quality control applied for each dataset. Thus, the results found here for the CONUS do not necessarily directly generalize to other regions.

Figure  presents KGE values for summer and winter for the 26 P datasets. The following observations can be made.

The spread in median KGE values among the datasets is much greater in winter than in summer. In addition, almost all datasets exhibit a greater spatial variability in KGE values in winter, as indicated by the wider boxes and whiskers. This is probably at least partly attributable to the lower quality of the Stage-IV dataset in winter .

Differences in median KGE values between uncorrected and gauge-corrected versions of P datasets ranged from -0.07 (GSMaP-Std Gauge V7) to +0.20 (CMORPH-CRT V1.0; Table ). GSMaP-Std Gauge V7 shows a large positive bias in the west (Fig. S2), suggesting that its gauge-correction methodology requires re-evaluation. The substantial improvements in median KGE for CHIRPS V2.0 (+0.13) and CMORPH-CRT V1.0 (+0.20) reflect the use of sub-monthly gauge data (5-day and daily, respectively). Conversely, the datasets incorporating monthly gauge data (IMERGDF V05 and WFDEI-GPCC) exhibited little to no improvement in median KGE (+0.05 and -0.01, respectively), suggesting that monthly corrections provide little to no benefit at the daily timescale of the present evaluation . These results, combined with the fact that several uncorrected P datasets outperformed gauge-corrected ones (Fig. ), suggest that a P dataset labeled as “gauge-corrected” is not necessarily always the better choice.

IMERG is NASA's latest satellite P dataset and is foreseen to replace the TMPA dataset (; Table ). The following main improvements were implemented in IMERG compared to TMPA: (i) forward and backward propagation of passive microwave data using CMORPH-style motion vectors ; (ii) infrared-based rainfall estimates derived using the PERSIANN-CCS algorithm ; (iii) calibration of passive microwave-based P estimates to the Combined GMI-DPR P dataset (available up to almost 70∘ latitude) during the GPM era and the Combined TMI-PR P dataset (available up to 40∘ latitude) during the TRMM era; (iv) adjustment of the Combined estimates by GPCP monthly climatological values to ameliorate low biases at high latitudes; (v) merging of infrared- and passive microwave-based P estimates using a CMORPH-style Kalman filter; (vi) use of passive microwave data from recent instruments (DMSP-F19, GMI, and NOAA-20); (vii) a 30 min temporal resolution (instead of 3-hourly); (viii) a 0.1∘ spatial resolution (instead of 0.25∘); and (ix) greater coverage (essentially complete up to 60∘ instead of 50∘ latitude).

These changes have resulted in considerable performance improvements: IMERGHH V05 performed better overall than TMPA-3B42RT V7 in terms of median KGE (0.62 versus 0.46), correlation (0.69 versus 0.59), bias (0.99 versus 1.09), and variability (1.05 versus 1.07; Figs. , , and a). The improvement is particularly pronounced over the northern Great Plains (Fig. a), where TMPA-3B42RT V7 exhibits a large positive bias (Fig. S2). In the west, however, there are still some small regions over which TMPA-3B42RT V7 performed better (Fig. a). Overall, our results indicate that there is considerable merit in using IMERGHHE V05 instead of TMPA-3B42RT V7 over the CONUS. Previous studies comparing (different versions of) the same two datasets over the CONUS , Bolivia , mainland China , southeast China , Iran , India , the Mekong River basin , the Tibetan Plateau , and the northern Andes reached largely similar conclusions.

ERA5 is ECMWF's recently released fourth-generation reanalysis and the successor to ERA-Interim, generally considered the most accurate third-generation reanalysis (; Table ). ERA5 features several improvements over ERA-Interim, such as (i) a more recent model and data assimilation system (IFS Cycle 41r2 from 2016 versus IFS Cycle 31r2 from 2006), including numerous improvements in model physics, numerics, and data assimilation; (ii) a higher horizontal resolution (∼0.28∘ versus ∼0.75∘); (iii) more vertical levels (137 versus 60); (iv) assimilation of substantially more observations, including gauge and ground radar P data (from 17 July 2009 onwards); (v) a longer temporal span once production has completed (1950–present versus 1979–present) and a near-real-time release of the data; (vi) outputs with a higher temporal resolution (hourly versus 3-hourly); and (vii) corresponding uncertainty estimates.

As a result of these changes ERA5-HRES performed markedly better than ERA-Interim in terms of P across most of the CONUS, especially in the west (Figs.  and b). ERA5-HRES obtained a median KGE of 0.63, whereas ERA-Interim obtained a median KGE of 0.55 (Fig. ). Improvements were evident for all three KGE components (correlation, bias, and variability). It is difficult to say how much of the performance improvement of ERA5 is due to the assimilation of gauge and radar P data. We suspect that the performance improvement is largely attributable to other factors, given that (i) the impact of the P data assimilation is limited overall due to the large amount of other observations already assimilated ; (ii) radar data were discarded west of 105∘ W for quality reasons ; and (iii) performance improvements were also found in regions without assimilated gauge observations (e.g., Nevada; Fig. b; , their Fig. 3). Nevertheless, we expect the performance difference between ERA5 and ERA-Interim to be less in regions with fewer or no assimilated gauge observations (i.e., outside the US, Canada, Argentina, Europe, Iran, and China; , their Fig. 3).

So far, only three other studies have compared the performance of ERA5 and ERA-Interim. The first study compared the two reanalyses for the CONUS by using them to drive a land surface model . The simulations using ERA5 provided substantially better evaporation, soil moisture, river discharge, and snow depth estimates. The authors attributed this to the improved P estimates, which is supported by our results. The second and third studies evaluated incoming shortwave radiation and precipitable water vapor estimates from the two reanalyses, respectively, with both studies reporting that ERA5 provides superior performance .

Ensemble modeling involves using outputs from multiple models or from different realizations of the same model; it is widely used in climate, atmospheric, hydrological, and ecological sciences to improve accuracy and quantify uncertainty . Here, we compare the P estimation performance of a high-resolution (∼0.28∘) deterministic reanalysis (ERA5-HRES) to that of a reduced-resolution (∼0.56∘) ensemble average (ERA5-EDA; Table ). The ensemble consists of 10 members generated by perturbing the assimilated observations as well as the model physics . The ensemble average was derived by equal weighting of the members.

Compared to ERA5-HRES, we found ERA5-EDA to perform similarly in terms of median KGE (0.62 versus 0.63), better in terms of median correlation (0.72 versus 0.69) and bias (0.96 versus 0.93), but worse in terms of median variability (0.80 versus 0.90; Figs. , , and c). The deterioration of the variability is probably at least partly due to the averaging, which shifts the distribution toward medium-sized events. The improvement in correlation is evident over the entire CONUS (Fig. c), and corresponds to a 9 % overall increase in the explained temporal variance, demonstrating the value of ensemble modeling. We expect the improvement to increase with increasing diversity among ensemble members .

IMERGHHE V05 and ERA5-HRES represent the state-of-the-art in terms of satellite P retrieval and reanalysis, respectively (Table ). Although the datasets exhibited similar performance overall (median KGE of 0.62 and 0.63, respectively; Figs.  and ), regionally there were considerable differences (Fig. d). Compared to ERA5-HRES, IMERGHHE V05 performed substantially worse over regions of complex terrain (including the Rockies and the Appalachians), in line with previous evaluations focusing on India and western Washington state . In contrast, ERA5-HRES performed worse across the southern–central US, where P predominantly originates from small-scale, short-lived convective storms which tend to be poorly simulated by reanalyses . The patterns in relative performance between IMERGHHE V05 and ERA5-HRES (Fig. d) correspond well to those found between TMPA 3B42RT and ERA-Interim (, their Fig. 4) and between CMORPH and ERA-Interim (, their Fig. 3d), suggesting that our conclusions can be generalized to other satellite- and reanalysis-based P datasets. Our findings suggest that topography and climate should be taken into account when choosing between satellite and reanalysis datasets. Furthermore, our results demonstrate the potential to improve continental- and global-scale P datasets by merging satellite- and reanalysis-based P estimates .

In addition to the (quasi-)global P datasets, we evaluated the performance of a state-of-the-art climate simulation for the CONUS (WRF; ; Table ). The WRF simulation has the potential to produce highly accurate P estimates since it has a high 4 km resolution, which allows it to account for the influence of mesoscale orography , and is “convection-permitting”, which means it does not rely on highly uncertain convection parameterizations . In terms of variability, WRF performed third best, being outperformed only (and very modestly) by the gauge-based CPC Unified V1.0/RT and MSWEP V2.2 datasets (Figs.  and ). In terms of bias, the simulation produced mixed results. WRF is the only uncorrected dataset that does not exhibit large biases over the mountainous west (Fig. S2). However, large positive biases were obtained over the Great Plains region, as also found by using the same reference data. In terms of correlation, WRF performed worse than third-generation reanalyses (ERA-Interim, JRA-55, and NCEP-CFSR; Figs.  and S1). This is probably because WRF is forced entirely by lateral and initial boundary conditions from ERA-Interim , whereas the reanalyses assimilate vast amounts of in situ and satellite observations . Overall, there appears to be some merit in using high-resolution, convection-permitting models to obtain climatological P statistics.