Uncertainty is generally defined as the state of being not completely confident or sure of something. The terms error, accuracy, bias, and precision are sometimes used to characterize the uncertainty. All these terms are quantifiable information about what is certain or uncertain. However, they are different from uncertainty by definition (Foody and Atkinson, 2003; Heuvelink, 1998; Loew et al., 2017). Error represents the difference between what is measured and its true value (JCGM, 2012). The true value is the exact value according to the theoretical definition of the variable being measured or estimated. If we perfectly know the true value, we have no measurement error, which eliminates uncertainty. Therefore, uncertainty stems from unknown true values and errors.

When a measurement can be repeated, its uncertainty can be described using probability distributions of the measured values or measurement errors compared to a reference (Montanari, 2007; Foody and Atkinson, 2003; Povey and Grainger, 2015). Figure 2 illustrates the relationship between uncertainty and other related terms when uncertainty is described by the probability distribution of the measured value or error. When accuracy is defined as the expectation (i.e., expected value) of overall error (e.g., Foody and Atkinson, 2003), bias (i.e., the difference between the expected value of an estimator and the true value of a parameter) is considered to be a measure of inaccuracy. Likewise, precision can be described using the standard deviation and variance of the probability distribution of measured values since they both denote error spread around the mean.

Some uncertainties cannot be described using a probability distribution function in modeling or measurement. These are called the “known, unquantifiable unknowns” (i.e., what we know to exist but are not able to quantify) and the “unknown unknowns” (i.e., what we do not know to “exist” because we cannot observe) (Povey and Grainger, 2015). The suitability of probability theory for quantifying uncertainty is widely debated in hydrological science (e.g., Beven, 2016; Nearing et al., 2016). Nearing et al. (2016) argue that there is epistemological uncertainty before selecting probability theory as the framework to estimate epistemic uncertainty (i.e., what we do not know certainly). Their uncertainty classification includes philosophical and linguistic types that are hardly quantifiable. Uncertainty assessment of satellite remote sensing data typically reports quantifiable errors but not unknown and unquantifiable errors (Povey and Grainger, 2015).

Raw satellite imagery undergoes a chain of processing and analysis (retrieval model) to generate useful data and information for applications (Fig. 3). ET is not directly measured by sensors but is derived from models; thus, it is considered to be high-level processing by data providers (ESA, 2021; NASA, 2021). The retrieval models of low-level data (e.g., radiance, vegetation indices) share common formulas and usually require only raw satellite images. High-level data like RS-ET rely on various models with different concepts, assumptions, and data sources. Therefore, the uncertainty of RS-ET data products is strongly linked to model uncertainty (from model conceptualization and parameters) and input data uncertainty.

RS-ET data are typically acquired during satellite passes over specific areas of interest, resulting in essentially instantaneous estimates of remote-sensing-based evapotranspiration (RS-ET). Because many operational applications necessitate ET estimates over longer time intervals, such as daily, 10 d, or monthly totals, various methods have been developed to upscale these instantaneous RS-ET estimates to daily values (Jiang et al., 2021). Moreover, the designated resolution and the return interval of satellites might not be suitable for operational applications; thus, gap-filling and spatial-downscaling steps are common in many RS-ET retrieval models.

Changes in the spatial and temporal scales

Scale (both spatial and temporal) is best described as a triplet of support (or grain), spacing, and extent. Support is the volume, shape, size, and orientation that a measurement represents. In the realm of RS, the spatial support of pixel values can be equivalent to the resolution of the RS image, which is the (average) size of its constituent pixels.

Uncertainty assessment refers to the estimation of quantifiable uncertainties. The uncertainty from input factors (e.g., parameters and input data) can be quantified with uncertainty analysis, also called uncertainty propagation or error propagation (Crosetto et al., 2001; Heuvelink, 1998; Wadoux et al., 2020). There are many techniques for uncertainty propagation, and their suitability depends on several factors, including the number of uncertain inputs, uncertainty distribution and correlation of input variables, and model linearity (Mohammadi and Cremaschi, 2022). For RS-based models, analytical techniques which are based on the propagation of moment formulas (Taylor, 1997) are often not suitable because these models have complex relationships, and input uncertainty is not always normally distributed. Numerical techniques are generally applicable to RS-based models (Heuvelink, 1998; Crosetto et al., 2001).

The contribution of each input factor to the total uncertainty in the model output is determined by sensitivity analysis (Crosetto et al., 2001; Saltelli et al., 2021). Such analysis is primarily used to identify the factors that contribute most to the model uncertainty (Saltelli et al., 2019). There are two main approaches to sensitivity analysis: local and global sensitivity analysis. Local sensitivity analysis defines the model's sensitivity to an input factor (e.g., parameter or variable) as the first-order partial derivative of the model with respect to this input factor (Saltelli et al., 2019). In contrast, global sensitivity analysis explores the whole variation range of the input factors (Razavi and Gupta, 2015).

Validation is often applied to confirm a data product's fitness for purpose instead of sensitivity analysis and uncertainty analysis of its retrieval model (Crosetto et al., 2001). The definition of validation in modeling is context dependent and has become more well defined over time (Bellocchi et al., 2011). Model validation does not prove that the model is true but rather proves that it is empirically adequate (Oreskes et al., 1994). A valid model is one that does not contain known or detectable flaws and is internally consistent rather than being an assertion of the actual reality. Meanwhile, validation of model results involves quantifying the accuracy compared to a reference (often in situ datasets), which proves the validity of the data for its intended application. In RS, validation often only refers to the data itself and not the model (Bayat et al., 2021; Loew et al., 2017; Wu et al., 2019a). Because RS-derived data products are model results, their validation depends on the quality and quantity of input parameters and the accuracy of auxiliary hypotheses that were used to derive them (Oreskes et al., 1994). Therefore, validating an RS-ET model does not imply that the model can be applied with any forcing data or settings to produce accurate outputs.

The academic electronic databases Web of Science and Scopus were searched (https://www.webofscience.com/, last access: 24 July 2023 and https://www.scopus.com/, last access: 24 July 2023) using the combination of the three search terms evapotranspiration, remote sensing, and uncertainty or their variants (Table 1). The terms transpiration and interception were not used since they only represent components of ET. Since different terms for satellite remote sensing, evaporation, and uncertainty can be used in the title and abstract, the variants of search terms were identified based on a set of 34 prior articles (Sect. S1 in the Supplement).

The search results were limited to publication dates from 2011 to 2021, and duplicates were removed. Only English articles (>99 % of results) that reported original research and were published in scientific peer-reviewed journals were considered. Review papers, conference proceedings, and gray literature were not included because they have different formats and provide limited details of the methods used for uncertainty assessment.

From the search results, we identified papers that attempted to assess the accuracy or uncertainty of one or more satellite remote-sensing-based estimations of terrestrial ET, either from model simulations or analysis-ready data products. The models of interest were diagnostic RS-ET models,

Diagnostic RS-ET models are static models that estimate ET at a single period of time (snapshot) using satellite data as the primary inputs for independent variables in the models.

Each included article was classified into categories based on methods, objectives of the study, and results (Table 2). The total number and percentage of research papers per category were then synthesized from the literature database, and the patterns and trends in assessing the uncertainty of RS-ET were discerned. In addition, the most common method for uncertainty assessment was identified, and articles that used this method were included in a meta-analysis to derive the typical range of uncertainty in RS-ET estimates.

In validation, RS-ET model results are compared to a reference method that is considered by the researcher to be the best or most valid measure. The choice of the reference method introduces subjectivity into the model evaluation (Melsen et al., 2019). In the case of RS-ET, three types of reference are typically used: (1) in situ measurements (N=572), (2) catchment water balance (N=83), and (3) output from models run with ground-based input data (N=9). Almost all articles that used the validation approach considered an in situ measurement as their reference (Fig. 7), while other types of reference data were much less considered.

Intercomparison is the second most widely used method (212 out of 676 studies). In intercomparison, the RS-ET estimates from multiple models are compared without assuming a superior one. This approach is mainly used to evaluate the relative uncertainty of a model compared to others (170 out of 212 studies). Intercomparison has also been used to evaluate other sources of uncertainty. For example, uncertainty from a change in spatial support can be evaluated by comparing model outputs using different input-upscaling methods (e.g., Ershadi et al., 2013; Sharma et al., 2016). Intercomparison has also been used to evaluate uncertainty due to the choice of input datasets (e.g., Long et al., 2011; Wang et al., 2016; Badgley et al., 2015).

Since the RS-ET datasets have both temporal and spatial dimensions, comparing RS-ET models or products is usually done by aggregating over one or two dimensions (i.e., resampling to a lower resolution). The simplest method of intercomparison involves aggregating ET estimates both temporally and spatially into one value (e.g., global annually averaged ET) and then comparing this value from different models or products (e.g., Mueller et al., 2013; Pan et al., 2020). Other methods of intercomparison involve comparing time series of spatially aggregated ET (e.g., monthly basin-scale ET). Aggregating over one of the two spatial dimensions is sometimes applied (e.g., Pan et al., 2020; Chen et al., 2019). The time series can also be aggregated by land cover classes (e.g., Weerasinghe et al., 2020) or climate zones (e.g., Trambauer et al., 2014), describing how RS-ET uncertainty varies under different conditions. For spatial intercomparison, temporally aggregated RS-ET maps can be compared visually (e.g., Weerasinghe et al., 2020) or by using simple map algebra (e.g., M. Jung et al., 2019). Only a few studies have applied metrics to evaluate the spatial similarity between two datasets, such as the spatial efficiency metric (SPAEF) (Stisen et al., 2021; H. C. Jung et al., 2019) and the degree correlation measure of spherical harmonic coefficients (López et al., 2017). None of these methods can characterize uncertainty in RS-ET fully; thus, combining them would provide a more comprehensive intercomparison.

Sensitivity analysis is the third most used approach in the reviewed literature but is only applied a in a small proportion of the reviewed studies (61 out of 676 articles). Out of these, only seven studies applied global sensitivity analysis (GSA). Sobol's (2001) method was applied to the parameters of the MODIS16 algorithm (Zhang et al., 2019), the TSEB model (Burchard-Levine et al., 2020), and three RS-ET models (PT-DTsR, MODIS16 algorithm, and PML) (Cao et al., 2021). This method was also applied to input variables of RS-ET models alone (e.g., Gomis-Cebolla et al., 2019). Elhag (2016) applied a similar variance-based sensitivity measure for the SEBS model but did not refer to Sobol's method. The extended Fourier amplitude sensitivity test has also been applied for GSA (García et al., 2013). This limited number of studies shows that the application of GSA to RS-ET models has been under-researched during the last decade despite the importance of GSA in environmental modeling (Saltelli et al., 2021).

The majority of articles that applied sensitivity analysis (54 out of 61) did not mention or apply a GSA method and thus were considered to be local sensitivity analyses (LSA). In most of these studies, LSA was done by changing one parameter at a time (one-at-a-time method) and calculating the ratio of the change in ET over the change in parameter (e.g., Long et al., 2011). In the reviewed articles, the one-at-a-time method has been implemented differently in terms of three factors: (1) the selection of parameters for LSA according to their importance, as judged by the researchers; (2) the range of values over which parameters are allowed to vary; and (3) the calculation of sensitivity for specific land covers. This suggests that LSA is influenced by the subjectivity of the researchers.

The uncertainties of key input datasets are sometimes evaluated by researchers in studies that assess uncertainty in RS-ET without explicitly being propagated to model outputs. This approach ranked fourth in the number of articles, with 20 out of 676. The key input datasets considered by researchers include air temperature, incoming shortwave radiation, incoming longwave radiation, wind speed, and land surface temperature (e.g., Vinukollu et al., 2011b; Pardo et al., 2014; Peng et al., 2016; Li et al., 2017). Input datasets were evaluated through validation with their in situ counterpart. Although other input datasets like vegetation indices are also important in RS-ET models, the in situ measurements of these are often not available for evaluation (Vinukollu et al., 2011b). Some of the forcing datasets of RS-ET models are not remotely sensed data but are products from atmospheric data assimilation systems (e.g., Global Land Data Assimilation System (GLDAS) and ECMWF atmospheric reanalysis (ERA)), which are sometimes provided with uncertainty estimates from data providers. Evaluating the input data provides crucial a priori information for propagating uncertainty to ET estimates. Furthermore, even if uncertainty propagation is not conducted, these assessments can help to identify sources of uncertainty in RS-ET; as the saying goes, “garbage in, garbage out”.

Only 8 out of 676 articles applied the uncertainty propagation approach, mainly the Monte Carlo methods (MCMs), to evaluate uncertainty in RS-ET. In MCMs, the model inputs are randomly sampled from their distributions and fed into the model to generate outputs repeatedly. The variance of the output distribution will then be considered to be the uncertainty in the model output (i.e., ET estimate) associated with the input variables. The limited application of uncertainty propagation can be attributed to its complexity and computational demand. Sensitivity analysis and uncertainty propagation are ideally carried out in tandem (Crosetto et al., 2001; Saltelli et al., 2019), but only five out of eight articles combined these approaches. The uncertainty propagation approach was also used for investigations beyond uncertainty quantification. For example, Talsma et al. (2018) used MCM to determine the uncertainties in ET partitioning (i.e., soil evaporation, interception, and transpiration) in three RS-ET models (MOD16, PT-JPL, and GLEAM) due to the relative uncertainty in the key variables.

In the reviewed studies, uncertainty propagation was done only at one or a few fixed locations by assuming the probability distribution of the input variables then simulating a range of ET values at these locations. This approach is computationally inexpensive but does not fully characterize uncertainties in a spatial field of ET. To fully quantify uncertainty in a scene, Cawse-Nicholson et al. (2020) introduced a method based on MCM and spatial–statistical models (Cressie, 1993). With this method, the probability distribution of ET per pixel in a satellite scene can be quantified and presented as percentile maps. This distribution was almost always non-Gaussian for all pixels in ET scenes, which means simple linear error propagation is not possible (Cawse-Nicholson et al., 2020). Future studies of RS-ET would benefit from the development of new methods to quantify uncertainty spatially.

The three-cornered hat method (TCH) (Premoli and Tavella, 1993) and triple collocation (TC) (Stoffelen, 1998; McColl et al., 2014) are related to the intercomparison approach in the sense that these techniques assess the relative uncertainty of three datasets without assuming one is the best. Therefore, these techniques are useful when there is a lack of a high-quality reference dataset. Both TC and TCH methods require a set of three datasets with the assumption that their errors are independent (Sjoberg et al., 2021). The difference between TCH and TC is that TC can only be used to assess uncertainties of uncorrelated datasets, while TCH can be used when there are correlations with proper constraints (Xu et al., 2019; Sjoberg et al., 2021). However, to date, few studies have evaluated uncertainties in RS-ET using TC (Miralles et al., 2011b; Barraza Bernadas et al., 2018; Khan et al., 2018; and Kibria et al., 2021) and TCH (Long et al., 2014; Xu et al., 2019; and He et al., 2020). The proportion of studies that used these methods is less than 2 % of the total reviewed articles and is not increasing (Fig. 6). This low adoption might be attributed to the limitations of these methods: (1) the lack of information about biases and the estimation of only random errors (e.g., RMSE, standard deviation, or variances), (2) the required conditions to achieve reliable error estimates (large samples, similar scales and magnitudes of errors between datasets) (Sjoberg et al., 2021), and (3) the reliability of TCH as an alternative to direct validation (Wu et al., 2019b).

Physical consistency can be understood as the plausibility that an ET estimate is consistent with the physical conditions or characteristics of the area it represents. A consistency check or physical validation was proposed by Zeng et al. (2015) as the final step in a general validation process for big remote sensing datasets. When there are limited reference data and ground-based measurements, physical validation is critical to assess the quality of data products (Blatchford et al., 2020). Although physical validation does not quantify uncertainty using metrics, it provides an evaluation of the data quality. This is useful for identifying the regions and conditions in which RS estimates are more uncertain and where more effort in direct validation approaches is required.

Only six studies in the selected literature have attempted to quantify this plausibility (Fig. 6), but they defined physical consistency differently. For example, Rwasoka et al. (2011) used FAO Penman–Monteith potential ET estimates as a threshold to decide whether ET estimates from the SEBS model were physically inconsistent. Blatchford et al. (2020) used the ET/P ratio and water availability (P-Q) to evaluate the physical consistency of the WaPOR ET product. López et al. (2017) developed a technique to assess the hydrological consistency of ET by transforming both ET and P data into spherical harmonics and then using spherical harmonic coefficients to calculate the degree correlation. These studies are not the same as validating RS-ET with P-Q or P-Q-dS/dt as discussed previously since these residuals were not considered to be the best reference of ET.

Another method to assess physical plausibility without an explicit water balance is through the Budyko curve. The Budyko curve describes the semi-empirical relationship between long-term ET and its limiting factors, i.e., precipitation and potential ET (PET), for river basins (Budyko, 1974). Koppa and Gebremichael (2017) validated the physical consistency of ET by calculating the RMSE of the Euclidean distance between the data points and the Budyko curve in ET/P and PET/P space. Weerasinghe et al. (2020) simply calculated the mean difference (bias) between RS-ET and Budyko-derived ET to evaluate which RS-ET product exceeds the energy and water limit defined by the Budyko curve. They also noticed that, if a data point does not align with the Budyko curve, it might also mean that the ET of the basin exceeds the water or energy limit, for example, due to human activities. Therefore, the interpretation of physical plausibility needs to consider the actual knowledge about water resources in the basin instead of focusing only on model-generated numbers.

Intercomparison studies sometimes lead to ensemble-mean products of all available products on the basis of the assumption that no model performs best so an ensemble of them would be preferable (Bhattarai et al., 2019; Elnashar et al., 2021). This approach has been used the least in the reviewed articles (Fig. 6). Some researchers have evaluated the uncertainty in an ensemble (a set) of RS-ET estimates from different models by calculating the average and range of all members in the ensemble (Vinukollu et al., 2011b; Elnashar et al., 2021; Guo et al., 2020). This approach is the same as the multi-model ensembles in climate modeling. The model structural uncertainty can only be quantified if independent models are sampled from the entire possible model space and avoid the over-representation of one model structure (Abramowitz and Gupta, 2008). For example, Vinukollu et al. (2011b) selected three RS-ET models, namely SEBS (Su, 2002), PM-Mu or MODIS16 (Mu et al., 2007), and PT-Fi or PT-JPL (Fisher et al., 2008), which are based on distinct equations used to estimate ET.

Using an ensemble of RS-ET estimates provides uncertainties of the ensemble but not of each individual member of the ensemble. Thus, some studies went further by merging the datasets of the ensemble and calculating the difference between this merged dataset with each ensemble member (Baik et al., 2018; Elnashar et al., 2021). If simply averaging all the ET products, the bias of different models can be canceled in regions where they perform differently but accumulated in regions where they perform in the same manner. Hence, the ensemble products may arguably produce better estimation in some areas (Yao et al., 2017) but not a better understanding of the physical processes and drivers needed to improve RS-ET (Yao et al., 2014; Zhang et al., 2016). Therefore, it is considered to be more useful to use the range of the ensemble to identify the outlier data products or the uncertainty of all data products.

The review shows that uncertainties in RS-ET estimates were assessed at all stages, from developing a new model to evaluating its data product. Uncertainty in RS-ET was assessed in the context of model implementation (34 % of reviewed articles), model development (13 % of all reviewed articles), model improvement (17 %), model evaluation (19 %), and product evaluation (16 %) (Fig. 8). Here, model implementation means that a pre-existing model was applied to new case studies or to achieve some specific research objective without considerable modification or further development of the model. The prominence of model implementation as the main objective in the reviewed articles could be due to a perceived need to assess the uncertainty of RS-ET estimates for each application despite previous validation. This is an important attitude in the research community since it helps to provide feedback on appropriate application and improvement of RS-ET models. Therefore, studies in the context of model implementation should not be overlooked.

The reviewed articles evaluated all sources of uncertainty as categorized in the theoretical framework (Fig. 3), with a strong focus on compound uncertainty. Figure 9 shows that the majority (406 out of 676) of reviewed articles assessed only compound uncertainty without disaggregating into other sources. The second largest set of articles assessed both compound uncertainty and the relative uncertainty of RS-ET estimates. Other sources of uncertainty are remarkably less evaluated in the selected literature. According to the number of articles in each set (Fig. 9), the level of interest in different sources of uncertainty can be ranked as follows: compound, relative, input data, model parameterization, change of spatial support, change of temporal support, and finally gap filling. This does not necessarily show the ranking of importance of the uncertainty sources but rather the availability of methods and the data needed to assess them.

The uncertainties due to temporal upscaling are affected by several factors related to location (Jiang et al., 2021). These factors includes vegetation cover, soil moisture (Gentine et al., 2007; Hoedjes et al., 2008), cloud coverage (as discussed in research by Van Niel et al., 2012), cloud frequency (as explored in studies by Xu et al., 2015), air pollution effects (as indicated in research by Zhang et al., 2013), the return interval of the satellite (Alfieri et al., 2017), the timing of the overpass (Jiang et al., 2021), and the number of instantaneous values used for upscaling (Liu, 2021). Consequently, applying a single temporal-upscaling method for the entire globe results in spatially varying uncertainties in RS-ET estimates.

Uncertainties in RS-ET estimates are specific for different spatial and temporal supports. The reviewed studies evaluated RS-ET uncertainties at spatial supports ranging from less than 100 m up to global and temporal supports ranging from sub-daily to annual (Fig. 10). Most studies evaluated RS-ET uncertainties at spatial supports of 500 m to 5 km (268 out of 676) and less than 100 m (191 out of 676). This can be attributed to the availability of RS datasets that are widely used to estimate ET, such as MODIS (250 m to 1 km) and Landsat (30 to 100 m). In the case of validation, the spatial support of uncertainty assessment was determined by the spatial support of the ground truth reference. For temporal support, uncertainty was mostly evaluated by daily ET (365 out of 676), although RS datasets provide observations at the time of the satellite overpass with a temporal resolution of 5–16 d. This shows that the temporal support of uncertainty assessment is driven more by practical needs and less by the availability of datasets.

Assessment of RS-ET uncertainties is not evenly distributed over the globe. The number of articles per country where uncertainties in RS-ET were assessed is shown in Fig. 11. Each article was tagged by the country where the sites of study are located. The highest number of articles assessed ET in China. Because the most common approach is validation and the most common reference used is EC measurements, ET was mainly assessed where there are EC stations (i.e., AmeriFlux, AsiaFlux, ChinaFlux, OzFlux, EuroFlux, FLUXNET). Even when the studies aimed to validate RS-ET globally, the estimated uncertainty is not universal since these networks do not cover many regions. These studies were also included in Fig. 11.

Based on its popularity, EC can be considered to be the de facto standard ET estimation approach for validation of RS-ET. However, this popularity is mainly driven by the number of publications in countries where EC towers are more densely distributed (e.g., China and the United States of America). In countries where there are very few or no EC towers available, the most common reference used for validation of RS-ET is the water balance method (Fig. 12). In a few countries in northern Africa and the Middle East, the most common method is to use the FAO-56 method (Allen et al., 1998) in combination with crop coefficients to estimate ground-based references for validation (e.g., Egypt and Iran).

The reviewed articles that assess uncertainty in RS-ET mainly report accuracy (RMSE), bias (mean error), and the goodness of fit with a reference dataset (R2) (Fig. 13). Although quantifiable uncertainty in measurement is theoretically represented as a probability distribution, this has rarely been done in the literature. The reviewed studies used a wide range of metrics to report their uncertainty assessment (33 metrics). Most studies used 3 metrics, while some used up to 12. Larger numbers of metrics provide more descriptions of uncertainty, but some metrics might be challenging to interpret.

Root-mean-square Error (RMSE) is the most widely used metric in the reviewed articles (531 out of 676 articles). Metrics related to RMSE include normalized RMSE (normalized by standard deviation) and relative RMSE (as a percentage of mean ET). Very few studies (17 articles) used modified RMSE to report more robust results, and few consider random errors and systematic errors, such as robust RMSE (Bisquert et al., 2016), systematic and unsystematic RMSE (Yebra et al., 2013), and biased and unbiased RMSE (Martens et al., 2017). RMSE has the unit of the estimates so it can be expressed in millimeters (mm) for ET or watts per square meter (W m-2) for latent heat flux. Therefore, to compare reported RMSEs between different studies, unit conversion is needed.

Inconsistent use of metrics such as R2 might cause misinterpretation of results, especially when comparing studies. For example, the second most used evaluation metric was referred to using many names including mean error, mean difference, bias error, or bias. Meanwhile, the coefficient of determination (R2) has the opposite issue in which the same term was used with different formulas. R2 is a measure of the goodness of fit for regression models. There are at least eight formulas for R2 in the literature (Kvålseth, 1985), but only one formula can be used for any type of model fitting (i.e., R12 in Kvålseth, 1985). Since many studies did not report which formula they used, we did not distinguish between different R2 formulas in Fig. 13. Nevertheless, we observed that at least four different formulas of R2 were used in the reviewed articles, including the squared coefficient of correlation (Table S4).

No matter which metrics are used, the validation metrics that compare estimates with references only represent the actual error if the reference is the absolute truth. This is never the case because in situ measurements and upscaling methods are never perfect. Wu et al. (2019a) suggested that validation should be performed in conjunction with uncertainty associated with in situ measurements and the statistical significance of performance metrics.

Although there are a large number of papers assessing the uncertainty of RS-ET data, only a few attempted to synthesize their results. For example, a review by Karimi and Bastiaanssen (2015) used meta-analysis (i.e., using statistical methods to synthesize the results of independent studies) to estimate the probability density function of mean absolute percentage error (MAPE) in 46 studies that validate RS-ET estimates in seasonal cycles. Kalma et al. (2008) summarized the relative error and RMSE of the RS-ET reported in 30 studies. These syntheses are limited in terms of the number of studies (<50) and the fact that the selection of studies was not systematic. Another limitation of synthesizing these results is that the selected studies used different validation data and field instruments, which do not have equivalent spatial support and accuracy.

Synthesizing results of the reviewed articles in this study will provide a useful reference for future studies to evaluate the results of RS-ET uncertainty assessment. For a meta-analysis, selected studies should use the same validation data and report the same metric; thus, we selected the most used validation data and metric. Since the majority of studies used EC flux towers and RMSE to report uncertainty (372 out of 676), we selected these studies for meta-analysis of reported RS-ET uncertainty. From 372 articles, 348 articles that reported the RMSE of RS-ET from validation with EC flux towers were included. The remainder were excluded because the RMSE was not reported in figures with extractable values (Fig. 4). RMSE values in units other than millimeters per day (mm d-1) were converted to millimeters per day (mm d-1) assuming a constant rate of ET over the temporal support. For example, 365 mm yr-1 was converted to 1 mm d-1, and 0.1 mm h-1 was converted to 2.4 mm d-1.

The reported RMSE values for daily ET (N=3167) range from 0.01 to 6.65 mm d-1, with a mean value of 1.18 mm d-1 (Table 3), which is comparable with RMSEs previously reported by Kalma et al. (2008). When converting RMSE values from the reported unit to a common unit of millimeters per day, the mean RMSE is the highest for the validation of instantaneous RS-ET (2.81 mm d-1) and the lowest for monthly (0.78 mm d-1). In general, studies with larger temporal support of validation have lower mean RMSEs in millimeters per day. For the validation at temporal supports of 3 h, 10 d, and weekly, less can be concluded due to the small number of studies and records. Overall, the decrease in RMSE with increasing temporal support is due to the averaging and corrective effect of temporal upscaling. Therefore, improving temporal-upscaling and gap-filling methods are crucial for reducing uncertainty in RS-ET estimates.

Figure 14 shows that very high RMSE values were mainly from validation approaches that used a single EC site. Validation using data from a greater number of EC sites tends to yield lower RMSE values. This might be attributed to the fact that, when papers only report average RMSE values across multiple EC sites, the average RMSE is lower than the highest individual RMSE. Moreover, the random errors at each site are likely to be uncorrelated or partially cancel each other out when averaging them, which further reduces the overall RMSE. As RMSE is inherently dependent on the scale of ET, sites with lower ET values or the practice of averaging ET across multiple sites are more likely to exhibit lower average RMSE values. Unfortunately, only a limited number of studies provided information on the relative RMSE or the average ET corresponding to RMSE values, which hindered the derivation of scale-independent RMSE values across all studies.

The control factors of ET and the uncertainty in their estimates are not the same globally (Zhang et al., 2016). As the distribution of validation sites is concentrated in regions where EC flux towers are available (Fig. 12), the results of the validation are thus not necessarily transferable to other areas. Therefore, when interpreting the uncertainty of RS-ET based on validation, we should consider the validation metrics at each site individually and the variation of these metrics among all locations.

The large range of RMSE values obtained from the meta-analysis can be explained by the diversity of reviewed studies in terms of models, resampling methods, and validation context (e.g., temporal scales, land cover, climate, amount of data). For example, some studies validate RS-ET estimates from global products, while others validate RS-ET estimates from models that were calibrated to reduce RMSE. Moreover, many studies reported the RMSE of latent heat flux (in W m-2 or MJ m-2 d-1) averaged from estimates at the time of satellite overpass. The accuracy of RS-ET varies at different times of the day due to weather conditions and is thus not representative of the entire day. We converted these values to millimeters per day (mm d-1) (Table 3) only for comparison between different temporal supports. The range of RMSEs presented in Fig. 14 and Table 3 should only be considered as a baseline for typical errors in RS-ET. Using only RMSE to compare RS-ET model performance across different studies or validation sites is not recommended.