The SMAP L4 soil moisture product (version 4; Reichle et al., 2019) is generated by assimilating the SMAP L1C Radiometer half-orbit 36 km Equal-Area Scalable Earth (EASE) Grid Tb observations (Version 4 SPL1CTB; Chan et al., 2016) into the CLSM. The SMAP Tb observations are assimilated at 3 h intervals using a spatially distributed, 24-member ensemble Kalman filter (Reichle et al., 2017b). The surface meteorological forcing data are from the global Goddard Earth Observing System (GEOS) Forward Processing atmospheric analysis (Lucchesi, 2013), with precipitation corrected using the daily, 0.5∘, gauge-based Climate Prediction Center Unified (CPCU) product (Xie et al., 2007). The L4 product provides global, 9 km, 3-hourly surface (0–5 cm), and root-zone (0–100 cm) soil moisture estimates along with related land surface fields and analysis diagnostics. For the present study, we aggregate all soil moisture estimates to daily averaged (00:00 to 23:59 UTC) data. The OL baseline is a model-only, ensemble CLSM simulation without the assimilation of SMAP Tb observations but otherwise using the same configuration, including perturbations, as in the L4 system (Reichle et al., 2021).

The SMAP L4 assimilation system includes a zero-order τ–ω forward RTM (De Lannoy et al., 2013) that converts SSM and surface soil temperature into L-band brightness temperature estimates. Selected parameters of the L4 RTM, including the microwave soil roughness parameter h, vegetation structure parameter τ, and the microwave scattering albedo ω, are calibrated using multi-angular L-band brightness temperature observations from the Soil Moisture Ocean Salinity (SMOS) mission (De Lannoy et al., 2014a). The L4 RTM parameterizes microwave soil roughness as a function of SSM (De Lannoy et al., 2013; their Eq. B1). Here, we use this parameterization to compute the 2017–2018 daily averaged microwave soil roughness estimates as one potential indicator of DA skill improvement (Sect. 2.3). The necessary parameters are obtained from L4 Land Model Constants output collection (10.5067/KGLC3UH4TMAQ; Reichle et al., 2018a). The L4 Analysis Update Data output collection includes RTM predictions of Tb and the assimilated SMAP Tb observations (10.5067/60HB8VIP2T8W; Reichle et al., 2018b).

To avoid the impact of seasonality, we perform our analysis using anomaly time series, derived by subtracting a seasonally varying (daily) climatology from each raw time series. The climatology of a given time series is obtained by sampling the mean value of all soil moisture estimates that fall within a 31 d moving window centered on a particular day of year. Moreover, L4 estimates of LE, sensible heat flux (H), and the climatological LAI inputs to CLSM and the RTM are obtained from the L4 Geophysical Data output collection (10.5067/KPJNN2GI1DQR; Reichle et al., 2018c). These datasets are also used to compute control factors to explain spatial variations in the DA skill improvement of the L4 system (Sect. 2.3).

In situ soil moisture measurements during 2017 and 2018 are collected from a national network of Chinese Automatic Soil Moisture Observation Stations (CASMOS) maintained by the Chinese Meteorological Administration (CMA; Han et al., 2017). In total, soil moisture measurements from 2474 separate stations across China, and covering different land use types, are collected. At each CASMOS site, frequency domain reflectometry-based instruments (DNZ1, DNZ2, and DNZ3) are used to record hourly volumetric soil moisture content within the following vertical depth ranges: 0–10, 10–20, 20–30, 30–40, and 40–50 cm below the surface. This instrumentation – DNZ1, DNZ2, and DNZ3 – is separately produced by Shanghai Changwang Meteorological Science and Technology Corporation (Shanghai, China), Henan Meteorological Science Research Institute and the 27th Institute of China National Electric Power Corporation (Zhengzhou, China), and China Huayun Technology Development Corporation (Beijing, China), respectively. From the above-mentioned instruments, the hourly estimates (at multiple depths) are then aggregated into daily values and linearly averaged (vertically) to produce 0–10 cm (SSM) and 0–50 cm (RZSM) in situ soil moisture measurements, which are subsequently used to validate the L4 and OL SSM (0–5 cm) and RZSM (0–100 cm) estimates. Note that Spearman's correlation rather than Pearson's correlation is used for L4 and OL validation because Pearson's correlation assumes linear consistency of the underlying variables and is more sensitive to outliers. By employing Spearman's rank correlation, we do not need to exclude soil moisture outliers and thus avoid introducing ad hoc thresholds that would define outliers. Nonetheless, we repeat the analysis based on Pearson's correlation (not shown) and find that the results are qualitatively consistent with the results using Spearman's correlation.

Ground observations within the same 9 km EASE grid were averaged for comparisons against the collocated 9 km L4 and OL soil moisture estimates. A total of 2287 individual 9 km EASE grid cells within China are included in the analysis. Among them, 92.35 % of grid cells contain one in situ site, 7.26 % contain two sites, seven grid cells contain three sites, and the remaining two grid cells contain four and five sites respectively. Figure 2 shows the number of in situ CASMOS sites within each 9 km EASE grid.

As discussed above, our hypothesis is that the efficiency of the SMAP L4 system will be sensitive to the ability of the ensemble-based L4 analysis in filtering errors that exist in CLSM, the RTM forecast Tb, and the assimilated SMAP Tb observations. We therefore consider two separate categories of factors that potentially control spatial variations in DA skill improvement. The factors are summarized in Table 1.

The first category represents a range of factors known to affect the skill of soil moisture estimates derived from the LSM (in this case, CLSM). The five control factors in this category are (1) the error in precipitation forcing, (2) the error in (input) LAI, (3) the error in (output) LE, (4) the magnitude of mean error in CLSM SSM–RZSM coupling strength, and (5) the presence of vertical variability in soil properties (defined as the difference in clay fraction across the vertical soil profile). Note that such variability represents a potential source of error because, with the exception of some surface-layer moisture transport parameters, CLSM assumes soil texture and associated soil parameters are vertically homogeneous within the soil column. However, the Harmonized World Soil Database (HWSD; FAO/IIASA/ISRIC/ISSCAS/JRC, 2012) often captures distinct vertical variations in soil properties, which are neglected by CLSM. Therefore the magnitude of vertical heterogeneity in soil texture may be an effective proxy for overall CLSM soil moisture accuracy. HWSD is selected due to its extensive use in soil science (De Lannoy et al., 2014b), and switching from HWSD to the high-resolution soil hydraulic and thermal properties dataset derived from the Global Soil Dataset for Earth System Models and SoilGrids (Dai et al., 2019) does not qualitatively change our conclusion or the importance ranking of vertical variability in soil properties (figure not shown). In addition, given the high specific surface area of clay and its high influence on soil structure and aggregation, the clay fraction is very important for soil moisture retention (Hillel, 1998), and thus the clay fraction is chosen over silt and sand fractions in the analysis. Besides, note that since LE and H are generally (strongly) anti-correlated, it is not appropriate to include both in a single random forest analysis – since including both would yield biased (high) regression weights for LE and H.

The second category contains three factors that affect radiative transfer modeling (RTM) and therefore DA updates. These include (1) estimates of the Tb innovation, namely the difference between SMAP Tb observations and RTM Tb simulations; (2) the magnitude of microwave soil roughness; and (3) the magnitude of LAI (as a proxy for the vegetation optical depth at microwave frequencies, which modulates the contribution of surface soil to the observed Tb).

The control factors take a variety of forms. Some factors are based on estimates of the errors fed into the L4 system, namely (1) the error in CLSM rainfall forcing data, (2) the error in SSM–RZSM coupling strength, (3) the vertical variability of clay fraction, (4) the SMAP L4 LAI error, (5) the output LE error, and (6) the error in Tb innovation. Other factors consist of the magnitude of the variable itself, namely the magnitude of microwave soil roughness and annual mean LAI. Note that LAI is used in both ways: LAI error is used to predict OL performance (because LAI is an important input into CLSM), while mean LAI is used to explain DA performance (because increased LAI is associated with decreased soil moisture information in microwave observations).

Note that the LAI used in the L4 system is a merged climatology from Moderate Resolution Imaging Spectroradiometer (MODIS) and Geoland data based on satellite observations of the normalized difference vegetation index (Mahanama et al., 2015; Reichle et al., 2017a). Therefore, to indicate the magnitude by which the LAI of each grid cell typically deviates from its long-term climatology, we use the temporal standard deviation for the anomaly time series of a benchmark LAI time series as a measure of the error in the LAI value used in the L4 system. This benchmark LAI is from the SPOT-VEGETATION (SPOT VGT) product and includes inter-annual variations (Sect. 2.3.3). Owing to the lack of reference Tb observations at similar satellite overpass times and locations, errors in Tb innovation are gauged using the time series standard deviation of the observation-minus-forecast (O-F) Tb residuals, which indicate the typical misfit between the RTM forecast Tb and the rescaled SMAP Tb observations. This rescaling process ensures zero-mean differences between Tb observations and forecasts and involves a seasonal multi-year-mean bias correction, which makes sure that the DA only corrects for errors in short-term and inter-annual variations and not for errors in the climatological seasonal cycles of the modeled soil moisture or other land surface fields. The standard deviation of the O-F Tb residuals measures the total error in Tb observation space.

The exact datasets and the metrics utilized for evaluating all eight control factors are summarized in Table 1.

The RZSM time series generally show decreased temporal dynamics relative to SSM. As a result, overestimated SSM–RZSM coupling tends to spuriously increase the (correlation-based) similarity of SSM and RZSM time series and, thereby, overestimate RZSM temporal variability. Therefore, analogous to the Kling–Gupta efficiency (Gupta et al., 2009), we define the SSM–RZSM coupling strength (CP) as 1CP=1-(R-1)2+(α-1)2, where R is Spearman's rank correlation between SSM and RZSM, and α is the ratio of temporal standard deviation of SSM to that of RZSM. The CP estimation is based on anomaly time series of both SSM and RZSM. A CP value of 1 represents the extreme case where RZSM is identical to SSM, i.e., a strongly coupled case. Likewise, a CP of 0 represents the opposing case of completely uncoupled time series. Cases with negative CP do not exist in this study.

Observed CP (CPobs) was based on comparisons between 0–10 cm surface and 0–50 cm root-zone in situ observations and used as a benchmark. In contrast, CP estimates of OL (CPOL) was based on the comparison of 0–5 cm surface and 0–100 cm root-zone estimates. Therefore, the surface versus root-zone storage contrast in the observation time series is less than that of the L4 estimates. This will likely cause the observed correlation between surface and root-zone time series to be systematically higher than the analogous vertical correlation calculation for L4 estimates. However, this bias is partially corrected for by the second term in Eq. (1) – since the observed α ratio will, by the same token, tend to be smaller (i.e. closer to 1) than α sampled from the L4 analysis. Such ability to compensate for vertical depth differences is a key reason we apply CP, rather than simple correlation, as a vertical coupling strength metric. Nevertheless, it should be noted that our main interest here lies in describing spatial variations in CPOL–CPobs, and care should be taken when interpreting raw CPOL–CPobs differences as an absolute measure of L4 vertical coupling bias.

The benchmark dataset of FLUXCOM LE described above contains error that is assumed to be of a similar order of magnitude as the L4 LE dataset it is applied to evaluate. Therefore, in an attempt to correct for the impact of this error, the LE error used here as a control factor is obtained via a double instrumental variable (IVd; Dong et al., 2019b) analysis approach that minimizes the spurious impact of random errors in benchmark datasets. As shown in Dong et al. (2019b), for the evaluation of two time series containing autocorrelated errors, IVd is more robust than a single instrumental-variable-based algorithm; therefore we apply IVd to evaluate the LE error.

IVd is a modified version of triple collocation (TC) analysis. In TC analysis (McColl et al., 2014), geophysical variables obtained from three independent sources (xt, yt, and zt) at time t are assumed to be linearly related to the true signal Pt as 2xt=αxPt+Bx+εx,t, where the αx is a scaling factor, Bx is a temporal constant bias, and εx,t is the zero-mean random error.

As opposed to the TC method, IVd uses only two independent products (x, y) to characterize geophysical data product errors. This method introduces two instrumental variables I, which is the lag - 1 time series of x, and J, which is the lag - 1 time series of y, respectively. 3It=αxPt-1+Bx+εx,t-14Jt=αyPt-1+By+εy,t-1 Therefore, assuming that the errors of two independent products are serially white, the covariance between instrumental variables and products can be written as follows: 5CIx=αx2LPP6CJy=αy2LPP, where C represents the covariance of the subscript products. For instance, CIx represents the covariance of x and its instrumental variable I. Variable LPP is the lag - 1 auto-covariance of the true signal. Combining Eqs. (5) and (6), the scaling ratio sivd of the two products x and y can be written as 7sivd=CIxCJy. Based on Eq. (7), their correlation with truth can be estimated as 8RPx2=CxysivdCxx9RPy2=CxyCyysivd. In this way, the error in the L4 LE (measured by IVd-based correlation with truth) can be estimated robustly using the FLUXCOM LE product described in Sect. 2.3.2.

A random forest (RF) regression approach is used to quantify and rank the importance of the eight control factors introduced above (Table 1) for describing spatial patterns in DA skill improvement for both SSM and RZSM estimates. The RF method is a supervised learning algorithm based on an averaged ensemble of decision trees (Breiman, 2001). Unlike linear regression approaches, RF can capture nonlinear interactions between the features and the target. In addition, the normalization (or scaling) of data is not necessary in RF application. Another advantage of the RF algorithm is that it can readily measure the relative importance of each feature on the estimates, which makes it highly suitable for an attribution analysis. Therefore, based on the output of RF, key control factors determining the skill improvement of SMAP DA are evaluated and ranked. The RF estimates are based on a 10-fold cross-validation approach.

Figure 3 maps validation results (i.e., Spearman's rank correlation of anomaly with in situ observations, R) for SMAP L4 and associated OL soil moisture estimates. The skill patterns for OL and L4 are, in general, quite spatially consistent. Both are characterized by an increasing trend of SSM estimation skill moving from northwestern to southeastern China (Fig. 3a and b) that matches the increasing density of the rain gauge network. In relative terms, the L4 product surpasses the baseline OL's SSM skill for 77 % of the 2287 9 km EASE grid cells containing ground observations – with a mean R increase of ΔR=0.056 [–] and mean relative improvement versus ROL of 14 %.

Similar spatial patterns are observed for RZSM skill. As with SSM, generally higher consistency with in situ RZSM measurements is found in southeastern China relative to northwestern China. However, relative to SSM, the benefit of SMAP data assimilation (i.e., L4) is reduced for RZSM, and the mean relative R improvement is only 7 % (ΔR=0.034 [–]) (compare Fig. 3e and f). This reduction is expected since assimilated SMAP Tbs are primarily sensitive to soil moisture conditions in the surface (0–5 cm) layer.

As described in Sect. 2.3, we select eight control factors that potentially influence the skill of SMAP L4 soil moisture estimates. Using the attribution analysis described in Sect. 2.6, these factors are used to explain the spatial variations in DA skill and DA skill improvement seen in Fig. 3. As a first step, this section examines the spatial patterns inherent in the eight control factors. Errors in the CLSM precipitation forcing are relatively higher in northwestern China (Fig. 4a), where the gauge density is generally sparser than in southern China. Among the factors representing CLSM structural errors, a predominantly negative bias is observed in SSM–RZSM coupling strength generally across China (i.e., lower CPOL compared to CPobs), while a very small number of grid cells show a positive coupling strength bias in eastern China (dark green dots in Fig. 4b). This is expected since the coupling strength generally decreases with coarser resolution; i.e., the vertical coupling strength of the model is assumed much lower than that of any single site. In addition, this may be partially attributed to layer depth differences, since CLSM represents surface and root-zone depths of 0–5 and 0–100 cm, respectively, whereas the corresponding in situ observations represent the 0–10 and 0–50 cm layers. Therefore, CPOL is likely to be systematically smaller than CPobs. In addition, the vertical variability of the clay fraction seems to show little spatial variation across China (Fig. 4c). With respect to CLSM LAI error, regions in southern China that have generally higher LAI show larger standard deviations in SPOT LAI time series (Fig. 4d and h). The IVd-based estimates of SMAP L4 LE error, which represent a potential control factor for water-balance errors in CLSM, generally show a low level of error across China (Fig. 4e).

For O-F Tb residuals describing RTM-related error, a higher standard deviation of O-F Tb residuals is observed in the North China Plain (Fig. 4f), which is very consistent in spatial distribution with areas displaying the highest and most significant SSM prediction improvement (Fig. 3c). This is expected, as mentioned above, because O-F Tb residuals are the basis for the soil moisture corrections (or increments) that are applied in the DA system as part of the L4 analysis. The 2017–2018 mean of soil roughness shows a relatively scattered spatial pattern (Fig. 4g), while the 2017–2018 mean LAI shows higher values in southwestern and southeastern China (Fig. 4h).