Introduction
Knowledge of precipitation plays an important role in the understanding of
the water cycle, and thus in the management of water resources (Sellers,
1997; Sorooshian et al., 2005; Wang et al., 2005; Ebert et al., 2007; Buarque et
al., 2011; Tapiador et al., 2012; Yong et al., 2012; Gao and Liu, 2013; Peng et
al., 2014a, b). However, precipitation data are not
available in many regions, particularly mountainous districts and rural
areas in developing countries. For example, northeast China, which plays an
important role in food production to support the country's population and is
also an industrial region with many heavy industries, frequently suffers
from drought, posing a threat to regional sustainable development. In such
areas, due to insufficient gauge observations, alternative precipitation
data are required for efficient management of water resources.
In recent years, implementation of gauge-based and remote satellite-based
precipitation products has become popular, particularly for ungauged
catchments (Artan et al., 2007; Jiang et al., 2012; Li et al.,
2013; Müller and Thompson, 2013; Maggioni et al., 2013; Xue et al.,
2013; Kneis et al., 2014; Meng et al., 2014; Ochoa et al., 2014). Numerous
precipitation products have been developed to estimate rainfall, for
example: Tropical Rainfall Measuring Mission (TRMM) products
(Huffman et al., 2007), Global Land Data Assimilation System
(GLDAS) precipitation products (Kato et al., 2007), Asian
Precipitation – Highly-Resolved Observational Data Integration Towards
Evaluation of Water Resources (APHRODITE) (Xie et al., 2007; Yatagai
et al., 2012), Precipitation Estimation from Remotely Sensed Information
using Artificial Neural Networks (PERSIANN) (Sorooshian et al.,
2000, 2002), and Global Satellite Mapping of Precipitation
product (GSMAP) (Kubota et al., 2007; Aonashi et al., 2009).
There are uncertainties in these products. Several studies have been carried
out to analyse the uncertainty of TRMM in high-latitude regions (Yong et
al., 2010, 2012, 2014; Chen et al., 2013a; Zhao and
Yatagai, 2014), but studies in northeast China are few. Evaluation of GLDAS
data has generally been limited to the United States and other
observation-rich regions of the world (Kato et al., 2007);
assessments and applications in other regions are rare (Wang et al.,
2011; Zhou et al., 2013). The APHRODITE, PERSIANN, and GSMAP products are
seldom evaluated in northeast China using basin-scale gauge data
(Zhou et al., 2008). Owing to the high heterogeneity of rainfall
across a variety of spatiotemporal scales, the uncertainty characteristics
of precipitation products are variable (Asadullah et al., 2008; Dinku et
al., 2008; Nikolopoulos et al., 2010; Pan et al., 2010). Thus, in northeast
China, it is essential to completely evaluate the applicability of these
precipitation products. In addition, it is also worth comparing the
performance of different precipitation computation recipes: for example, the
artificial neural network function used in PERSIANN, the histogram matching
approach used in TRMM3B42, and the cloud motion vectors used in
GSMAP-MVK+, because the inter-comparison could reveal the strategies that
could be used to obtain more accurate precipitation data.
Researchers have implemented precipitation products in discharge simulations
and reported discharge uncertainties (Hong et al., 2006; Pan et al.,
2010; Serpetzoglou et al., 2010). Also, many uncertainty analysis approaches
have been introduced to quantify the uncertainty (Beven and Binley,
1992; Freer et al., 1996; Kuczera and Parent, 1998; Beven and Freer,
2001b; Peters et al., 2003; Heidari et al., 2006; Kuczera et al., 2006; Tolson
and Shoemaker, 2007; Blasone et al., 2008; Vrugt et al., 2009). In these prior approaches, one of the popular methods is the
generalized likelihood uncertainty estimation (GLUE) technique, introduced
by Beven and Binley (1992). This approach outputs probability
distributions of model parameters conditioned on observed data, and the
uncertainties in model inputs are represented by uncertain parameters.
Similar to GLUE, Hong et al. (2006) proposed a Monte Carlo-based method
to quantify uncertainty in hydrological simulations using satellite
precipitation data, in which flow simulation uncertainty is represented by
ensemble simulation results.
In addition to individual contributions from hydrological models and
precipitation data, the interactions between precipitation products and
hydrological models also contribute to uncertainty in simulated discharges.
However, to the best of our knowledge, the previous studies have not
quantified the respective contributions of precipitation products,
hydrological models, and their interactions to the total discharge simulation
uncertainty.
The overall objectives of this paper are (1) to investigate the
applicability of six fine-resolution precipitation products using both
statistical and hydrological evaluation methods in a small river basin in
northeast China; (2) to propose a framework to quantify the contributions of
various uncertainties from precipitation products, hydrological models, and
their interactions to uncertainty in simulated discharges. The precipitation
products investigated are TRMM3B42, TRMM3B42RT, GLDAS/Noah
(GLDAS_Noah025SUBP_3H), APHRODITE, PERSIANN, and GSMAP-MVK+. Two hydrological models of different complexities – a
water- and energy-budget-based distributed hydrological model (WEB-DHM)
(Wang et al., 2009a, b, c) and a
physically based semi-distributed hydrological model TOPMODEL (Beven
and Kirkby, 1979) – were employed to investigate the influence of
hydrological models on discharge simulations. The respective uncertainties
from precipitation products, hydrological models, and the combined
uncertainties from the interactions between products and models are
quantified using a global sensitivity analysis approach, i.e. the analysis
of variance approach (ANOVA). A river basin with a series of 8-year data is
used to demonstrate the methodology.
The paper is organized as follows. Section 2 introduces the study region,
precipitation products, hydrological models, and the proposed framework.
Section 3 presents the statistical evaluation results. Hydrological
evaluations and the implementation of the proposed framework are given in
Sect. 4. Discussion is given in Sect. 5. Summary and conclusions are
presented in Sect. 6.
Precipitation products.
Product
Spatial
Temporal
Areal coverage
Start date
Type
resolution
resolution
TRMM3B42
0.25∘
3h
Global 50∘ N–S
1 Jan 1998
PR + IR + MW + gauge + HM
TRMM3B42RT
0.25∘
3 h
Global 50∘ N–S
1 Mar 2000
IR + MW
GLDAS/Noah
0.25∘
3 h
Global 90∘ N–60∘ S
24 Feb 2000
IR + MW + gauge
GSMAP-MVK+
0.1∘
1 h
Global 60∘ N–S
1 Mar 2000
IR + MW + CMV
PRRSIANN
0.25∘
3 h
Global 60∘ N–S
1 Mar 2000
PR + IR + MW + ANN
APHRODITE
0.25∘
1 day
60–150∘ E,
1 Jan 1961
gauge
15∘ S–55∘ N
to 2007
PR: precipitation radar; IR: infrared estimation; MW: microwave estimation;
HM: histogram matching; CMV: cloud motion vectors; ANN: artificial neural network.
Biliu basin: (a) the location of Liaoning province within China;
(b) the location of Biliu basin within Liaoning province; (c) the distributions
of rain gauges, discharge gauge, automatic weather stations, digital
elevation model, and diagrammatic 0.25∘ precipitation cells; and
(d) diagrammatic description of downscaling the 0.25∘ precipitation cells
to 300 m × 300 m cells, and retrieving the 300 m × 300 m
cells located within the basin boundary.
![]()
Materials and methodology
Biliu basin
Biliu basin (2814 km2), located in the coastal region between the China
Bohai Sea and the China Huanghai Sea, covers longitudes 122.29 to
122.92∘ E and latitudes 39.54 to 40.35∘ N.
This basin is characterized by a snow, winter dry, and hot summer climate
(Koppen climate classification) and the average annual temperature is
10.6 ∘C. Summer (July to September) is the major rainy season.
There are 11 rainfall stations and one discharge gauge which have historical
data from January 2000 to December 2007. The average elevation is 240 m.
The gauge distribution in Biliu is shown in Fig. 1. The basin slopes
vary from 0 to 38∘. Land-use data are obtained from the USGS
(http://edc2.usgs.gov/glcc/glcc.php). The land-use types have been
reclassified to SiB2 land-use types for this study (Sellers et al., 1996). There are six
land-use types, with broadleaf and needle leaf trees and short vegetation
being the main types. Soil data are obtained from the Food and
Agriculture Organization (FAO, 2003) Global data product, and there are two
types of soil in the basin: clay loam Luvisols and loam Phaeozems.
Precipitation products
The selected precipitation products are shown in Table 1. These data are all
freely available. In these selected precipitation products, APHRODITE is
wholly based on gauge data; TRMM3B42 and GLDAS are remote satellite
estimations with gauge data corrections; while others are remote satellite
estimations without gauge data corrections. Remote-based precipitation
estimation has many weaknesses; e.g. microwave estimation could miss
convective rainfall and typhoon rain because of its sparse time interval
resolution; infrared estimation has a higher time interval resolution, but
it cannot penetrate thick clouds. Ground rain-gauge-based interpolation
products are limited by interpolation algorithms, gauge density, and gauge
data quality (Xie et al., 2007). The details of data sources used in
each precipitation product can be found in Table 1. The detailed
introductions of these products are as follows.
TRMM is a joint mission between NASA and Japan Aerospace Exploration Agency
designed to monitor and study tropical rainfall (Kummerow et al.,
2000; Huffman et al., 2007). Three instruments – a visible infrared
radiometer, a TRMM microwave imager, and a precipitation radar - are employed
to obtain accurate precipitation estimation. The TRMM precipitation radar is
the first space-based precipitation radar and operates between 35∘ N
and 35∘ S. Outside this band, the microwave imager is used
between 40∘ N and 40∘ S, and the visible infrared
radiometer data are used between 50∘ N to 50∘ S. Usually
the precipitation radar is considered to give the most accurate estimation
from satellite, and data from it are often used for calibration of passive
microwave data from other instruments (Ebert et al., 2007). The
post-real-time product used in this study is the TRMM3B42, which utilizes
three data sources: the TRMM combined instrument estimation using data from
both TRMM precipitation radar and the microwave imager; the GPCP monthly
rain gauge analysis developed by the Global Precipitation Climatology
Center; and the Climate Assessment and Monitoring System monthly rain gauge
analysis. TRMM3B42 applies an infrared to rain rate relationship using
histogram matching, while TRMM3B42RT merges microwave and infrared
precipitation estimation.
PERSIANN is a product that, using an artificial neural network function,
estimates precipitation by combining infrared precipitation estimation and
the TRMM combined instrument estimation (which assimilates with TRMM
precipitation radar and microwave data). GSMAP-MVK+ uses microwave and
infrared precipitation data together and combines cloud motion vectors to
generate fine-resolution precipitation estimation.
The Global Land Data Assimilation System (GLDAS) project is an extension of
the existing and more mature North American Land Data Assimilation System
(Rodell et al., 2004). It integrates satellite- and
ground-based data sets for parameterizing, forcing, and constraining a few
offline land surface models for generating optimal fields of land surface
states and fluxes. At present, GLDAS drives four land surface models: Mosaic
(Koster and Suarez, 1992), Noah (Chen et al., 1996; Betts et al.,
1997; Koren et al., 1999; Ek, 2003), the Community Land Model
(Dai et al., 2003), and the Variable Infiltration Capacity model
(Liang et al., 1994). Among them, the GLDAS/Noah Land Surface
Model product (GLDAS_NOAH025SUBP_3H) has a 3 h
0.25∘ × 0.25∘ resolution, which is desirable for
basin-scale research. The GLDAS precipitation data combine microwave and
infrared data, and also assimilate gauge observations.
Criteria for accuracy assessment
Uncertainties of precipitation products are evaluated on the basis of
basin-averaged rainfall observations. Four evaluation criteria are used in
rainfall amount error assessment: correlation coefficient (CC), root mean
square error (RMSE), Nash–Sutcliffe coefficient of efficiency (NSE), and
relative bias (RB). These are calculated as follows:
RMSE=∑i=1nXpi-Xoi2n12NSE=1-∑i=1nXpi-Xoi2∑i=1nXpi-Xo‾2RB=∑i=1nXpi-∑i=1nXoi∑i=1nXoi×100%,
where Xoi represents observed data; Xpi represents estimated data;
n is the total number of data points. A perfect fit should have CC and NSE
values of 1. The lower the RMSE and RB, the better the estimation. These
comparison criteria have been used by many studies (Ebert et al.,
2007; Wang et al., 2011; Yong et al., 2012), so they are used in this study.
Probability distributions by occurrence and volume are also analysed, which
can provide us with the information on the frequency and on the product
error dependence on precipitation intensity (Chen et al., 2013a, b).
The critical success index (CSI), probability of detection (POD),
and false alarm ratio (FAR) are used to quantify the ability of
precipitation products to detect observed rainfall events. These are defined
as follows:
CSI=HH+M+FPOD=HH+MFAR=FH+F,
where H is the total number of hits; M is the total number of misses; F is the
total number of false alarms (Ebert et al., 2007; Su et al.,
2008). A perfect detection should have CSI and POD values equal to 1 and a
FAR value of 0.
Hydrological models and data
WEB-DHM
The distributed biosphere hydrological model, WEB-DHM (Wang et al.,
2009a, b, c), was developed by coupling a
simple biosphere scheme (Sellers et al., 1986) with
a geomorphology-based hydrological model (Yang, 1998) to describe water,
energy, and CO2 fluxes at a basin scale. WEB-DHM has been used in
several evaluations and applications (Wang et al., 2010a, b, 2012; Shrestha et al., 2014).
WEB-DHM input data include precipitation, temperature, downward solar
radiation, long-wave radiation, air pressure, wind speed, and humidity. With
the exception of precipitation, all input data are obtained from automatic
weather stations. There are three automatic weather stations near Biliu, and
observations from these are obtained from the China Meteorological Data
Sharing Service System (downloaded from http://cdc.cma.gov.cn/home.do).
Hourly precipitation data are downscaled from daily rain gauge observations
using a stochastic method (Wang et al., 2011). Hourly temperatures are
calculated from daily maximum and minimum temperatures using a temperature model
(Parton and Logan, 1981). The estimated temperatures are also
further evaluated using daily average temperature. Downward solar radiation
is estimated from sunshine duration, temperature, and humidity using a hybrid
model (Yang et al., 2006). Long-wave radiation is obtained from the
GLDAS/Noah (Rodell et al., 2004). Air pressure is
estimated according to altitude (Yang et al., 2006). These
meteorological data are then interpolated to 300 m × 300 m model
cells through an inverse-distance weighting approach. Because of the
elevation differences among model cells and meteorological gauges, the
interpolated surface air temperatures are further modified with a lapse rate
of 6.5 K km-1. Gauge rainfall data are also interpolated to 300 m × 300 m
model cells and basin-averaged gauge rainfall data are calculated on
the basis of interpolation results. In addition to the above, the leaf area
index and fraction of photosynthetically active radiation data are obtained
from level-4 MODIS global product MOD11A2. The digital elevation model (DEM) is
from the NASA SRTM (Shuttle Radar Topographic Mission) with a resolution of
30 m × 30 m. We resampled the resolution to 300 m in model
calculation to reduce computation cost, while the model processed finer DEMs
(30 m grid) to generate sub-grid parameters (such as hillslope angle and length).
TOPMODEL
TOPMODEL is a physically based, variable-contributing area model of basin
hydrology which attempts to combine the advantages of a simple lumped
parameter model with distributed effects (Beven and Kirkby, 1979).
Fundamental to TOPMODEL's parameterization are three assumptions:
(1) saturated-zone dynamics can be approximated by successive steady-state
representations; (2) hydrological gradients of the saturated zone can be
approximated by the local topographic surface slope; and (3) the
transmissivity profile whose form declines exponentially with increasing
vertical depth of the water table or storage is spatially constant. On the
basis of the above-mentioned assumptions, the index of hydrological
similarity is represented as the topographic index, ln(a/tanβ),
for which a is the area per
unit contour length and β is the local slope angle. More detailed
descriptions of TOPMODEL and its mathematical formulation can be found in
Beven and Kirkby (1979). TOPMODEL has been popularly utilized in research across
the world (Blazkova and Beven, 1997; Cameron et al., 1999; Hossain and
Anagnostou, 2005; Bastola et al., 2008; Gallart et al., 2008; Bouilloud et al.,
2010; Qi et al., 2013), because of its relatively simple model structure. The
input data of TOPMODEL mainly include basin-averaged precipitation and
topographic data which can be estimated from DEM.
Diagrammatic flowchart of the proposed framework for quantification
of uncertainty contributions to ensemble discharges simulated using
precipitation products on the basis of the analysis of variance (ANOVA) approach.
![]()
The proposed framework
Figure 2 shows the diagrammatic flowchart of the proposed framework for
quantification of uncertainty contributions to ensemble discharges simulated
using precipitation products. This framework includes four parts:
(a) selection of precipitation products; (b) selection of hydrological models;
(c) ensemble of discharge simulations using the hydrological models and
precipitation products; and (d) quantification of individual and interactive
contributions using the analysis of variance (ANOVA) approach including
contributions from precipitation products, hydrological models, and interactions between models and products. Because the spatial resolution of
selected precipitation products does not correspond with WEB-DHM model
cells, the following procedures were carried out for basin-averaged rainfall
calculations: (1) resampling 0.25 or 0.1∘ precipitation product
grids into 300 m × 300 m cells (the grid size used in WEB-DHM
simulations); (2) calculating basin-averaged precipitation using 300 m
precipitation product grids located within the basin boundary. Diagrammatic
descriptions of these procedures are shown in Fig. 1d. Because WEB-DHM needs
hourly input data, for the 3 h resolution precipitation products, we
assumed rainfall is uniformly distributed within each 3 h period. For
daily resolution products, we used the same approach as downscaling observed
precipitation data. This downscaling approach may affect uncertainty in
simulated discharge. However, Wang et al. (2011) have already
successfully applied the downscaling approach, and shown that the
influence is negligible.
The total ensemble uncertainty Y is the variance of discharges. To relate Y to
the uncertainty sources, the superscripts j and k in Yj,k represent a
combination of precipitation product j and hydrological model k:
Yj,k=Pj+Mk+PMj,k,
where P represents the effect of jth precipitation product, M represents the
effect of kth hydrological model, and PM represents the interaction effect. In
this study, j varies from one to six, and k varies from one to two. Details of
the quantification are explained in the follow sections.
Subsampling approach
ANOVA could underestimate variance when the sample size is small
(Bosshard et al., 2013). To reduce the effect of the sample
size, Bosshard et al. (2013) proposed a subsampling method,
which was used in this paper. In the subsampling method, the superscript j in
Eq. (7) is replaced with g(h, i). According to Bosshard et al. (2013),
in each subsampling iteration i, data from two products should be
selected out of all the six products, and thus 15 combinations can be
obtained. Therefore, the superscript g becomes a 2 × 15 matrix:
g=11…122…44523…634…566.
Uncertainty contribution decomposition
Based on the ANOVA theory (Bosshard et al., 2013), total error
variance (SST) can be divided into sums of squares due to the individual
effects as
SST=SSA+SSB+SSI,
where SSA is the error contribution of precipitation products, SSB is the
error contribution of hydrological models, and SSI is the error contribution
of their interactions.
The terms can be estimated using the subsampling procedure as follows:
SSTi=∑h=1H∑k=1KYg(h,i),k-Yg(o,i),o2SSAi=K⋅∑h=1HYg(h,i),o-Yg(o,i),o2SSBi=H⋅∑k=1KYg(o,i),k-Yg(o,i),o2SSIi=∑h=1H∑k=1KYg(h,i),k-Yg(h,i),o-Yg(o,i),k+Yg(o,i),o2,
where symbol o indicates averaging over a particular index; H is the
number of precipitation products (six in this study) and K is the number of
hydrological models (two in this study). Then the variation fraction
η2 is calculated as follows:
ηprecipitation2=1I∑i=1ISSAiSSTiηmodel2=1I∑i=1ISSBiSSTiηinteraction2=1I∑i=1ISSIiSSTi.
η2 has a value between 0 and 1, which represent 0 and 100 %
contributions to the overall uncertainty of simulated discharges
respectively. I equals 15 in this study. As shown in Eqs. (14)–(16), the
subsampling approach is necessary because it guarantees that every
contributor has the same denominator I. This same denominator makes sure that
the inter-comparison among precipitation contribution, model contribution,
and interaction contribution is free of influence from the sampling number
of precipitation products and hydrological models.
Statistical evaluations
Daily and monthly scales
Comparison of precipitation product data and gauge observations at a daily
scale is shown in Fig. 3. Observations are shown on the x axis and
precipitation product data are shown on the y axis. Four criteria, RMSE, CC,
NSE, and RB, are also shown. GSMAP-MVK+ is the best product and PERSIANN
has the poorest performance with respect to RMSE and NSE. GSMAP-MVK+ is
also the best with respect to CC, while GLDAS has the poorest performance
with a CC value of 0.55. With respect to RB, APHRODITE performs best and
GSMAP-MVK+ the second best, while TRMM3B42RT the least best with an RB
value of -38 %. None of the products can outperform others in terms of all
the statistical criteria. This may be due to the different limitations of
satellite sensors and inverse algorithms of precipitation products. This
situation shows that the selection of the best precipitation products is difficult.
Scatter plots of basin-averaged precipitation products versus gauge
observations at a daily scale.
![]()
Scatter plots of basin-averaged precipitation products versus gauge
observations at a monthly scale.
![]()
TRMM3B42RT and TRMM3B42 underestimate precipitation amounts. This
underestimation may be because convective rainfall always happens in summer
in northeast China (Shou and Xu, 2007a, b; Yuan et al., 2010), and
indicates the limitation of TRMM algorithms in high-latitude regions with
convective rainfall. This type of rainfall has a large rainfall amount
within a short time period and, therefore, cannot be captured by microwave
imager. This type of rainfall may also have a thick cloud that is
impenetrable by infrared (Ebert et al., 2007). Thus microwave and
infrared estimation could underestimate rainfall. Compared with TRMM3B42RT,
TRMM3B42 provides an improvement in RB. This improvement may be attributed
to the assimilation with gauge data and histogram matching. Compared with
APHRODITE and GSMAP-MVK+, TRMM3B42 has low accuracy as represented by RB.
This implies that the retrieval algorithm used by TRMM3B42 still needs to be
improved with respect to RB. The reason why APHRODITE outperforms TRMM3B42
is that APHRODITE is a gauge-based product. The fact that GSMAP-MVK+ outperforms
TRMM3B42 in terms of RB may be due to the cloud motion vectors it uses.
Compared with GSMAP-MVK+, GLDAS/Noah precipitation shows low accuracy in
all the criteria even though they use similar data sources: IR and MW.
Comparison of precipitation product data and gauge observations at a monthly
scale is shown in Fig. 4. Here, the APHRODITE product (Fig. 4d) performs
best based on RMSE, CC, NSE, and RB. GLDAS/Noah is the poorest in terms of
RMSE and NSE. With respect to CC, GLDAS and TRMM3B42 are equally poor, with
CC values of 0.81. The results also show that PERSIANN overestimates
precipitation amount, while Li et al. (2013) found PERSIANN
underestimates rainfall in south China. This may be attributed to the
different latitudes of the study regions.
Figure 5 shows time series of average monthly precipitation data against gauge
observations during the period 2000–2007. Each curve represents a different
precipitation product. GLDAS data (Fig. 5a) seriously underestimate high
rainfall. Similarly, TRMM3B42RT underestimates peak precipitation intensity
also. Comparatively, APHRODITE, PERSIANN, TRMM3B42, and GSMAP-MVK+ have
better performances.
Time series plots of basin-averaged precipitation product values
versus gauge observations at a monthly scale.
![]()
Inter-annual basin-averaged monthly precipitation.
![]()
Inter-annual evaluations
Figure 6 shows the inter-annual average monthly precipitation. Each curve
represents a different data product. PERSIANN overestimates in all the
12 months, while others underestimate, especially during the summer. This may
result from the artificial neural network function and limitations of
infrared and microwave estimation. APHRODITE data are relatively close to
observations. Compared with TRMM3B42RT, TRMM3B42 is better, which indicates
that the gauge corrections and histogram-matching used by TRMM3B42 impact
positively on accuracy. During the summer, discrepancies between products
become larger. With a decrease of rainfall magnitude, the discrepancies
between products reduce. This information implies that the differences in
precipitation estimation are related to precipitation magnitudes:
the larger the rainfall magnitudes, the greater the differences.
Probability distribution evaluations
Figure 7 shows the cumulative probability distribution function (CDF) by
occurrence (CDFc) and by volume (CDFv) for precipitation products.
Probabilities are shown on the y axis, and the x axis shows rainfall intensity
with a 1 mm day-1 interval log space.
PERSIANN is the best by both occurrence and volume. However, for CDFc,
TRMM3B42RT is the least best, and, for CDFv, TRMM3B42RT, and GLDAS/Noah are
comparable and worse than others. All precipitation products overestimate
occurrence and volume probabilities except rainfall intensities of larger
than 63 and 53 mm day-1 for occurrence and volume probabilities,
respectively. This may be because the precipitation products overestimate
the intensity of some heavy rainfall (recall the results in Sect. 3.1).
The results differ from those of Li et al. (2013), in which
PERSIANN has the poorest performance. This may result from differences in
study region (in the study of Li et al. (2013), south China was studied).
Probability distributions of the six precipitation products by
occurrence (CDFc) and volume (CDFv).
![]()
False alarm ratio, probability of detection, and critical success
index for the six precipitation products.
![]()
Contingency statistics
Figure 8 shows the false alarm ratio, probability of detection, and critical
success index for each precipitation product.
PERSIANN has the highest false alarm ratio among the products, while
TRMM3B42RT has the lowest. The false alarm ratio of TRMM3B42 is larger than
TRMM3B42RT, which indicates that the gauge corrections and histogram-matching used by TRMM3B42 do not provide positive effects on false alarm
ratio and may give rise to uncertainty in false alarm ratio. GSMAP-MVK+
has a lower false alarm ratio than TRMM3B42.
No obvious trends are observed for the false alarm ratio overall (compared
with the probability of detection and critical success index), which means
the false alarm ratio dependence on rainfall magnitude is weak. However,
Chen et al. (2013a) found the false alarm ratios of TRMM3B42 and
TRMM3B42RT to increase with an increase in rainfall intensity. The
differences are attributed mainly to observed data. In the study of
Chen et al. (2013a), national rain gauge data were employed, whereas
in this study more detailed basin data are used.
Among all selected products, GLDAS/Noah has the lowest probability of
detection and critical success index during periods of high rainfall
intensity, while APHRODITE retains a high probability of detection and
critical success index. This is because APHRODITE uses gauge observations,
and implies that the APHRODITE algorithm is effective. PERSIANN has
comparable probability of detection with APHRODITE. The critical success
index of GSMAP-MVK+ is also comparable with APHRODITE. Compared with
TRMM3B42RT, TRMM3B42 has greater probability of detection and comparable
critical success index. This information implies that the retrieval algorithm of
TRMM3B42 provides positive effects on probability of detection, but no
obvious positive impacts on critical success index.
WEB-DHM parameters.
Symbol (unit)
Brief description
Basin-averaged
value
Ks (mm h-1)
Saturated hydraulic conductivity for soil surface
26.43
Anik
Hydraulic conductivity anisotropy ratio
11.49
Sstmax (mm)
Maximum surface water storage
42.75
Kg (mm h-1)
Hydraulic conductivity for groundwater
0.36
α
van Genuchten parameter
0.01
n
van Genuchten parameter
1.88
Observed and simulated flows using WEB-DHM and TOPMODEL from 2000 to
2007: (a), (c), and (e) are daily, monthly, and inter-annual simulations using
WEB-DHM respectively; (b), (d), and (f) are daily, monthly, and inter-annual
simulations using TOPMODEL respectively.
![]()
TOPMODEL parameters.
Name (unit)
Description
Lower
Upper
Calibration
bound
bound
SZM (m)
form of the exponential
0.01
0.04
0.019
decline in conductivity
LNT0 (m2 h-1)
log value of effective
-25
1
-11.911
lateral saturated
transmissivity
RV (m h-1)
hill slope routing
2000
5000
2608.4
velocity
SRmax (m)
maximum root zone
0.001
0.01
0.006
storage
SR0 (m)
initial root zone deficit
0
0.01
0.005
TD (m h-1)
unsaturated zone time
2
4
2.885
delay per unit deficit
Decreasing trends are observed for all products in terms of probability of
detection and critical success index, matching the results of Chen et
al. (2013a) for TRMM3B42 and TRMM3B42RT. This indicates that probability of
detection and critical success index have relatively strong dependence on
rainfall magnitude, and implies that microwave and infrared precipitation
estimation may have relatively strong dependence on rainfall magnitude in
terms of probability of detection and critical success index.
Hydrological evaluations
Assessment of hydrological models
WEB-DHM was calibrated against observed discharges of Biliu. Six main
parameters were selected to calibrate using a trial and error approach due
to the model's computational burden. Model parameter multipliers were
calibrated, similar to the study by Wang et al. (2011). The trial and
error approach has two steps. First, all the multiplier values are set to 1
which represents the default parameter values from the Food and
Agriculture Organization (FAO, 2003) and the SiB2 model. Second, the
multiplier values are varied until acceptable discharge simulation accuracy is
obtained. The calibrated parameter values are listed in Table 2. The
simulated daily, monthly, and inter-annual results are shown in Fig. 9a, c, and e.
TOPMODEL uses basin-averaged parameter values, and these parameter values
are estimated by experience or observation. However, these methods do not
give precise parameter values. Therefore, the parameter values are
considered as uncertain and provided with ranges based on experience
(Beven and Kirkby, 1979; Beven and Freer, 2001a, b; Peters et al., 2003).
Six parameters of TOPMODEL were calibrated using the dynamically dimensioned
search algorithm (Tolson and Shoemaker, 2007), and the results are
given in Table 3. The simulated daily, monthly, and inter-annual results are
shown in Fig. 9b, d, and f.
Note that the parameters of TOPMODEL and WEB-DHM were calibrated using
observed precipitation data, and the accuracy of simulated discharges was
validated using gauge observations. Comparison with the rainfall–runoff
model parameter values reported for the case study catchment in previous
research shows that the parameter values are appropriate (Qi et al., 2013, 2015, 2016).
Daily-scale discharges
Figures 10 and 11 display scatter plots of discharges during the period
2000–2007 simulated using WEB-DHM and TOPMODEL against gauge observations at
a daily scale. Two criteria, NSE and RB, are shown. It should been noted
that the start dates are different for precipitation products, and observed
data were used when product data are not available: from 1 January 2000 to
29 February 2000 for TRMM3B42RT, GSMAP-MVK+, and PERSIANN; from 1 January 2000
to 23 February 2000 for GLDAS/Noah. These time periods were not
considered for accuracy comparison.
Scatter plots of simulated discharges with WEB-DHM against gauge
observations at a daily scale.
![]()
In the case of WEB-DHM simulations, the best NSE (0.41) corresponds with
APHRODITE (Fig. 10d), while the best value for RB (1 %) corresponds with
GLDAS/Noah. In the case of TOPMODEL simulations, the best NSE (0.41)
corresponds with APHRODITE, and the best value for RB (-24 %) corresponds
with APHRODITE also. Although the best NSE is the same for both WEB-DHM and
TOPMODEL simulations and the corresponding product is also the same, there is a
large difference in the best RB values. At the daily-scale precipitation
amount evaluation, the least best RB is -38 %, corresponding with
TRMM3B42RT (Fig. 3c). However, in WEB-DHM discharge simulation, the least
best RB (218 %) corresponds with PERSIANN, and, in the TOPMODEL simulation,
the least best RB (-62 %) corresponds with TRMM3B42RT. These differences
stem from differences in hydrological models and interactions between
hydrological models and precipitation product data.
All RB criteria at the daily-scale precipitation evaluations (recall the
results in Fig. 3) are amplified by TOPMODEL, while in the case of WBE-DHM,
some are amplified and the others are decreased. For example, for GLDAS and
PERSIANN, the RB criteria at the daily-scale precipitation evaluations are
-27 and 28 %, but they are -50 and 31 % in TOPMODEL simulations;
they are 1 and 218 % in WEB-DHM simulations. These differences result
from the influence of hydrological models and interactions between
precipitation products and hydrological models. These results reveal that a
hydrological model can amplify uncertainties in input data but also reduce
uncertainties, which may be due to the non-linear runoff generation process
in hydrological models. This finding is consistent with the research by
Yong et al. (2010).
Scatter plots of simulated discharges with TOPMODEL against gauge
observations at a daily scale.
![]()
Monthly scale discharges
Figures 12 and 13 display scatter plots of discharges during the period
2000–2007 simulated using WEB-DHM and TOPMODEL against gauge observations at
a monthly scale.
Scatter plots of simulated flows with WEB-DHM against gauge observations
at a monthly scale.
![]()
In the case of WEB-DHM, the best NSE and RB values are 0.73 and 1 %, which
correspond with TRMM3B42 and GLDAS respectively. In the case of TOPMODEL,
they are 0.58 and -24 %, corresponding with PERSIANN and APHRODITE
respectively. The combination of WEB-DHM and TRMM3B42 shows a satisfactory
performance, with NSE and RB values of up to 0.73 and -7 %, even though
TRMM3B42 is not the best in monthly scale precipitation data evaluation.
This reveals the influence of different combinations of hydrological models
and precipitation data on discharge simulation, implying that accurate
discharge simulation does not solely depend on the accuracy of a precipitation product.
At the monthly scale, although APHRODITE is the best precipitation product
and WEB-DHM model has better performance than TOPMODEL in calibration
(Fig. 9c and d), the combination of APHRODITE and WEB-DHM is not better in the
discharge simulation, which can be shown by comparing Fig. 12d with Fig. 13d
(the RB and NSE of WEB-DHM and APHRODITE combination are -37 % and 0.5,
but they are -24 % and 0.51 for the combination of TOPMODEL and
APHRODITE). This could be due to the interactive influence between
hydrological models and precipitation products, and implies that the
interactions between models and products could be large and have a big
influence on discharge simulations. In addition, comparison of Fig. 12d and b
shows that discharge simulation of APHRODITE is worse than TRMM3B42,
even though APHRODITE is the best precipitation product in terms of all the
selected criteria at a monthly scale precipitation amount evaluation. This
information shows that better precipitation products do not guarantee better
discharge simulations. These results imply that, although the
satellite-based precipitation products are not as accurate as gauge-based
products in rainfall amount estimation, they could have a better performance
in discharge simulations if the combination of precipitation product and
hydrological model is good.
Scatter plots of simulated discharges with TOPMODEL against gauge
observations at a monthly scale.
![]()
Inter-annual average monthly discharges.
![]()
Inter-annual average monthly discharges
Figure 14 shows the inter-annual average monthly discharges of all selected
precipitation products during the period 2000–2007. In the case of TOPMODEL,
PERSIANN agrees well with gauge observations, and all products underestimate
discharges in August. In the case of WEB-DHM, GLDAS data and TRMM3B42 data
have a better performance than other data but, with the exception of
PERSIANN, all products underestimate peak discharge in August. The
simulation results show huge differences even though Fig. 9e and f show that
TOPMODEL and WEB-DHM have almost the same performance using observed data;
this is because of the impacts of interactive influence between hydrological
models and precipitation products.
Uncertainty source quantification
All above results suggest that discharge simulations are influenced by
precipitation products, hydrological models, and interactions between
hydrological models and precipitation products. Thus it is essential to
quantify the respective influence. Figure 15a and b show contributions of
precipitation products, hydrological models, and their interactions to
uncertainties in monthly average discharges and different flow quantiles
respectively. Figure 15b shows quantiles computed at a daily time step. The
contributions of uncertainty sources are represented by stripes.
Figure 15a shows that precipitation data contribute most of the uncertainty in
discharges, and contribute more than hydrological models. Interactions
between hydrological models and precipitation products have large
contributions, at a similar level to those from hydrological models. In
summer (July to September), the contribution of precipitation data is less
than most other months except March. However, the uncertainty in
precipitation intensity increases in summer (recall the results in
Sect. 3.2). In non-summer months except March, the uncertainty contribution from
precipitation products is larger than in summer. These differences maybe
result from the non-linear propagation of uncertainty through hydrological
models. In March, the contribution of hydrological models is larger than in
other months, which may result from the decrease in influences of
interactions and precipitation products, and from the non-linear influence of
the hydrological models.
Figure 15b shows that, for small discharges (smaller than 25 % quantile
which corresponds to an observed discharge value of 1.79 m3 s-1) and large
discharges (larger than 99 % quantile which corresponds to an observed
discharge value of 157 m3 s-1), hydrological models contribute most of the
uncertainties. For middle-magnitude flows (between 25 and 99 %
quantiles), precipitation products contribute the majority, and the
contribution of interactions is not negligible and of similar magnitude to
the contribution from hydrological models. The contribution of interactions
is larger for middle-magnitude flows than for small and large discharges.
The different contributions of interactions for various magnitude flows may
be because different magnitude rainfall data could trigger different
hydrological processes (Herman et al., 2013). Small
discharges mainly come from base flows which are relatively stable and do
not need much rainfall to be triggered, and large discharges are mainly
controlled by overland flows when heavy precipitation occurs. Middle-magnitude discharges consist of contributions from base flows, lateral
subsurface flows, and overland flows, and can be triggered by rainfalls of
various magnitudes – thus interactions are more variable.
Contributions of uncertainty sources to (a) average monthly
discharges and (b) discharge quantiles based on daily-scale simulated results.
![]()
Although heavy rainfall data have high uncertainty (recall the results in
Sect. 3.1), precipitation products have a small contribution to
uncertainty in large discharges (Fig. 15b). This implies that the
uncertainty in high precipitation is compensated by the high non-linearity in
hydrological models.
In this study, because hydrological model parameters were calibrated using
gauge observations, the hydrological model parameter uncertainty was not
considered. Although the uncertainty contribution results in this study may
not be transferable to other basins, the proposed framework provides a
useful tool for quantifying uncertainty contributions in discharge
simulations using precipitation products.
Discussion
The spatial variations in precipitation are not considered in this study.
The study region is a small river basin, as shown in Fig. 1; there are only
11 grids inside the basin boundary for the precipitation products with a
spatial resolution of 0.25∘. Within a grid of 0.25∘, there are
no differences in precipitation amount between the 300 m × 300 m
grids used in hydrological models, and differences exist at the level of
0.25∘ grids only. Sapriza-Azuri et al. (2015) suggested that the
spatial variability of precipitation has little influence on rapidly
responding river discharges; this is the case in this study basin because the flow
transport time from the most upper part of the basin to the downstream
discharge gauge is 6 h, which is shorter than the daily and monthly time
steps of discharges investigated. Therefore, the spatial distributions of
precipitation products with a resolution of 0.25∘ in the relatively
small river basin have little influence on the simulated discharges.
However, the assumption of uniform distribution can be regarded as another
uncertainty source against spatial variability, and its influence can be
assessed using the proposed uncertainty quantification framework. This will
allow us to compare the relative contributions of the assumption to those
from other sources such as hydrological models, which will be investigated
using a much larger river basin in future work.
In addition to improving the accuracy of precipitation products, a good
coalition could help to achieve the performance in discharge simulations.
Our approach provides a way to assess the different coalitions, i.e. the
overall uncertainties in simulated discharges from different combinations of
hydrological models and precipitation products. More precipitation products
and hydrological models should be included and tested in future work.
It should be noted that other input data including temperature, downward
solar radiation, long-wave radiation, air pressure, wind speed, and humidity
may also have uncertainties. However, Fig. 9 shows that the simulated
discharge data are acceptable, particularly at monthly and inter-annual
scales using these data. Research has shown that the land surface
temperatures are highly accurate compared with MODIS satellite land surface
temperature observations (Wang et al., 2011; Qi et al., 2015). Thus, the
uncertainties from the other inputs are not considered in our case study river basin.
In this study, the parameter values calibrated using gauge observations are
not tuned to a specific product. That is, there is little compensation by
model parameters for the errors in input precipitation data. The differences
in modelling accuracy mainly result from the different representations of
hydrological processes. That is, the errors in precipitation products are
primarily compensated by the different representations of model processes.
Summary and conclusions
This research assesses the applicability of six precipitation products with
fine spatial and temporal resolutions at a high-latitude region in northeast
China, using both statistical and hydrological evaluation methods at
multi-temporal scales. A framework is proposed to quantify uncertainty
contributions of precipitation products, hydrological models, and their
interactions to simulated discharges. These products are TRMM version 7
products (TRMM3B42 and TRMM3B42RT), GLDAS, APHRODITE, PERSIANN, and
GSMAP-MVK+. The fully distributed WEB-DHM and semi-distributed TOPMODEL
were employed to investigate the influence of hydrological models on
simulated discharges. The results show the uncertainty characteristics of
the six products, and reveal strategies that could improve precipitation
products. This information could provide references for future precipitation
product development. The proposed framework can reveal hydrological
simulation uncertainties using precipitation products; thus it provides useful
information on precipitation product applications. The following conclusions
are presented on the basis of this study.
First, at a daily scale, selecting the best precipitation products is very
difficult, while, at a monthly scale, APHRODITE has the best performance in
terms of NSE, RB, RMSE, and CC, and also retains a high probability of
detection and critical success index. This information implies that the
APHRODITE algorithm is effective, and APHRODITE could be a very good data
set to refine and validate satellite-based precipitation products.
Second, GSMAP-MVK+ shows a huge advantage, and is better than TRMM3B42 in RB,
NSE, RMSE, CC, false alarm ratio, and critical success index, while PERSIANN
is better than TRMM3B42 in probability of detection and precipitation
probability distribution estimation. At present, the NASA Global
Precipitation Measurement (GPM) mission combines the artificial neural
network function of PERSIANN and precipitation radar-matching of TRMM
Multi-satellite Precipitation Analysis. However, the above finding implies
that incorporating the GSMAP-MVK+ estimation approach into GPM could be useful as well.
Third, it is found that, although high uncertainty exists in heavy rainfall,
hydrological models contribute mostly to the uncertainty in extreme
discharges. This may result from the non-linear propagation of uncertainty
through hydrological models, and implies that high uncertainties in extreme rainfall do not mean high
uncertainties in extreme discharges.
Fourth, interactions between hydrological models and precipitation products
contribute a lot to uncertainty in discharge simulations, and interactive
impacts are influenced by discharge magnitude. Because of interactive
effects, for hydrological models with similar performances in calibration,
using the same precipitation products for discharge simulations does not
provide a similar level of accuracy in discharge simulations, and in fact
very different predictions could be obtained. In addition, this finding
implies that only considering precipitation products or hydrological model
uncertainties could result in overestimation of precipitation product
contribution and hydrological model contribution to discharge uncertainty.
Fifth, a good discharge simulation depends on a good coalition of a
hydrological model and a precipitation product, and a better precipitation
product does not necessarily guarantee a better discharge simulation. This
suggests that, although the satellite-based precipitation products are not
as accurate as the gauge-based product, they could have better performance
in discharge simulations when appropriately combined with hydrological
models. It should be noted that this finding should be further tested with
more river basins, in particular large river basins accounting for spatial
variability in precipitation products.
In the future, calculating deterministic discharge simulations considering
precipitation product uncertainties and hydrological model uncertainties
should be studied because the above results show that product uncertainties
and model uncertainties all are important. In addition, recalibrating
hydrological models using precipitation products may reduce the interactive
influence between hydrological models and precipitation products on
simulated discharges, and this may explain why recalibration can improve
discharge simulation accuracy. This should be verified in future work.
Further, future research is encouraged to incorporate the GSMAP-MVK+
estimation approach into GPM because of the good performance of GSMAP-MVK+.