The Microwave Imaging Radiometer with Aperture Synthesis (MIRAS) onboard SMOS provides multi-angle Tb data, with a nominal (3 dB) spatial resolution of 43 km and a global coverage approximately every 3 days (at either 06:00 or 18:00 local time, i.e., ascending or descending half-orbits, separately). The most recent version (v620) of the SCLF1C Tb data is used. Observations are retained for further processing only (a) in the alias-free zone, (b) when the data are not contaminated by point source radio frequency interference (RFI) or tails thereof, (c) when the values fall within the range 100–320 K, and (d) when valid data are available for both horizontal (H) and vertical (V) polarization. The flag for snapshot RFI is not activated, because it is currently too sensitive (R. Oliva and Y. Kerr, personal communication, 2016). After the initial screening, we correct the L1 Tb values for geometric and Faraday rotation and for atmospheric and reflected extraterrestrial radiation using Modern-Era Retrospective Analysis for Research and Applications (MERRA) version 2 MERRA2; background fields. The resulting Tb values at the bottom of the atmosphere are then binned into 41 evenly spaced angular bins with the center angle ranging from 20 through 60∘. Next, the data are regridded from the 15 km discrete global grid (DGG) on which they are posted to the 36 km cylindrical Equal-Area Scalable Earth (EASEv2) grid , and the data are screened for excessive sub-36 km heterogeneity (spatial standard deviation > 7 K), which is indicative of open water bodies or RFI. Tb values for a given 36 km EASEv2 grid cell are computed only if at least two valid DGG observations are available.

From these preprocessed Tb data, two datasets are derived for assimilation: (i) a seven-angle Tb dataset, with incidence angles θ= [30, 35, 40, 45, 50, 55, 60∘] , and (ii) a fitted Tb dataset from which only the Tb at a 40∘ incidence angle is used to mimic the single-angle nature of SMAP Tb observations. We refer to these datasets as Tb_7ang and Tb_fit, respectively. Tb_fit data are only retained when the fitting error is less than 5 K and a minimum of 15 data points contribute to the entire fitted angular signature, with at least 5 data points above and below the 40∘ incidence angle and at least 10 data points in the incidence angle interval between 30 and 50∘.

The SMOS SM retrievals are extracted from the SMUDP2 product v552. Because this product version ends in early May 2015, we limit our study period to 1 July 2010–1 May 2015. (The reprocessed v620 version of the SM retrievals was not yet available at the time we conducted the experiments.) The SMOS retrieval algorithm simultaneously retrieves soil moisture and vegetation opacity, by fitting multi-angle Tb observations at both H- and V-polarization with simulations of the L-band Microwave Emission of the Biosphere Model L-MEB,. Based on the quality information provided within the SMOS products, the SM data are retained only if (a) all retrieved variables fall within a realistic range (0–0.6 m3 m-3 for soil moisture), (b) the SM uncertainty estimated by the SMOS retrieval algorithm is less than 0.1 m3 m-3, (c) the RFI probability for both H- and V-polarization is less than 0.3, and (d) SM retrieval flags are not raised for high topographic complexity, high urban fraction, high open water fraction, sea ice, coastal areas, and high total electron content. Further screening for frozen temperature and snow is based on GEOS-5 model output (Sect. ). After the regridding from the 15 km DGG grid to the 36 km cylindrical EASEv2 grid, the data are screened for excessive sub-36 km heterogeneity (spatial standard deviation > 0.2 m3 m-3). SM values for a given 36 km EASEv2 grid cell are computed only if at least two valid DGG observations are available.

The land data assimilation system used here employs the GEOS-5 catchment land surface model CLSM;, along with an L-band tau-omega radiative transfer model RTM;. The CLSM simulations use GEOS-5 parameters similar to those used in the SMAP L4_SM product, and are forced with 1/2∘ × 2/3∘ GEOS-5 forcing data from MERRA bilinearly interpolated to the model grid. The study domain covers most of North America, with the northwestern corner at (125∘ W, 55∘ N) and the southeastern corner at (60∘ W, 24∘ N).

The computational elements are the 36 km EASEv2 grid cells. The land model computation time step is 7.5 min, and output is saved at 3 h intervals. At each grid cell, the surface soil moisture content (sfmc, 0–5 cm) and root-zone soil moisture content (rzmc, 0–100 cm) are diagnosed based on three prognostic variables: catchment deficit (catdef), root-zone excess (rzexc), and surface excess (srfexc). Similarly, the surface (skin) temperature is diagnosed from the prognostic land surface temperatures across the saturated (tc1), unsaturated (tc2), and wilting (tc4) sub-grid areas. Finally, the soil temperature (tp1 for the topmost layer) is diagnosed from the prognostic ground heat content (ght1 for the top layer). An overview of the model variables is given in and .

The L-band tau-omega RTM converts the 36 km CLSM soil moisture and temperature simulations into 36 km L-band Tb estimates when the soil is not frozen or covered with snow, when precipitation is less than 10 mm day-1, and where the open water fraction is less than 5 %. For each 36 km grid cell, key parameters of the RTM are estimated by minimizing Eq. (B.1) in , using a 5-year history of SMOS v620 Tb data, and computing observation predictions (see below) at the footprint scale. Specifically, all 36 km grid cells within one footprint area are initially assigned the same set of RTM parameters, while the dynamic background information is spatially variable. For each 36 km grid cell, the calibration estimates a spatially homogeneous set of RTM parameters for the entire associated footprint area, and the resulting values are assigned to the central (and typically dominant) 36 km grid cell only. For the forward calculation of the Tb observation predictions during the data assimilation, all 36 km pixels have a unique set of RTM parameters. The RTM is calibrated using all 5 years of available Tb data and aims at minimizing climatological biases. The data assimilation is performed over the same 5 years and aims at addressing random (or short-term) errors. The methodology is very similar to that in , but with the difference that, here, the RTM does not simulate atmospheric contributions (because the Tb observations are now a priori corrected for atmospheric contributions) and the observation predictions are now spatially aggregated using a realistic (but approximate) antenna pattern.

For the computation of differences between SMOS observations and footprint-scale model simulations in the RTM calibration and for the computation of the “observation-minus-forecast” (O–F) residuals in the assimilation system (Sect. , Fig. ), the modeled 36 km soil moisture or Tb simulations are aggregated to the footprint scale by spatial convolution with weights given by an approximation of the SMOS antenna pattern. We also refer to these spatially aggregated model estimates as “observation predictions”. The SMOS antenna pattern is approximated by a two-dimensional Gaussian function containing 50 % of the signal within a circle with a radius of 20 km. The simulations outside a radius of 40 km are discarded in the computation of the footprint-scale estimates.

The number of 36 km EASEv2 grid cells included in one footprint area varies with latitude. The circular footprint shape is preserved everywhere on the globe. In contrast, the shape of the EASEv2 grid cells projected on the globe varies with the latitude, with an aspect ratio of 1 at 30∘ (north–south) latitude, larger than 1 towards the poles and less than 1 towards the Equator. Therefore, at higher latitudes multiple EASEv2 grid cells with the same latitude and various longitudes belong to one circular footprint, whereas towards the Equator, several EASEv2 grid cells with the same longitude and various latitudes contribute to the footprint. Overall, the difference between single 36 km simulations and footprint-scale values is small, but the number of valid Tb observation predictions at the footprint scale is reduced, because of the increased likelihood of finding a 36 km grid cell with a non-negligible water fraction, snow amount, or precipitation within the footprint area.

The assimilation results are evaluated using independent in situ measurements of surface and root-zone soil moisture from two sparse networks across the US: the US Natural Resources Conservation Service Soil Climate Analysis Network SCAN; and the US Climate Reference Network USCRN;. Surface soil moisture measurements are taken at approximately 5 cm depth. Root-zone soil moisture measurements are a weighted average of measurements at 5, 10, 20, and 50 cm depth, with respective weights of 0.1, 0.1, 0.27, and 0.53. Given the difference in spatial support between these point measurements and the 36 km gridded model and assimilation results, the skill is quantified in terms of anomaly time series correlation (anomR) and unbiased root-mean-square difference RMSDub;, using all 3 h forecast and analysis time steps in the period 1 July 2010–1 May 2015, excluding times when the soil is frozen (top layer soil temperature < 274.15 K) or snow covered (snow water equivalent > 0 kg m-2). The anomaly correlation is based on anomaly time series obtained by subtracting a multi-year smoothed climatology from both the simulations and in situ observations. Note that the assimilation and open-loop simulations have, by design, the same climatological variability; the assimilation only corrects for random errors. Metrics at a single site are only calculated if at least 200 data points are available. Skill metrics across an entire network are calculated by clustering the sites within SCAN and USCRN to avoid densely sampled areas dominating the validation metrics and to ensure realistic confidence intervals . The number of clusters is estimated a priori after prescribing an average cluster radius of 3∘, which approximately reflects the autocorrelation length of large-scale topographic and meteorological phenomena, or of large-scale soil moisture patterns . The actual size of the clusters that results from the clustering algorithm varies strongly in space.

For both Tb and SM retrieval assimilation, a spatially distributed (or three-dimensional, 3-D) ensemble Kalman filter EnKF; is used. This system simultaneously assimilates multiple spatially distributed observation sets, using horizontal and vertical error covariance structures, to update the simulations at each 36 km model grid cell. The details of the Tb assimilation system are explained in and differ only in that the observations are here associated with a spatially variable antenna pattern reaching out to a radius of 40 km.

During the model integration, a data assimilation step is activated every 3 h. All the SMOS observations yi collected within 1.5 h of the analysis time i are assimilated simultaneously to update the forecasted state x^k,ij- at location k as follows:

x^k,ij+=x^k,ij-+Kk,i[yij-y^ij-], with j denoting the ensemble member, Kk,i the Kalman gain, yij the perturbed observations, y^ij-=hi(x^ij-) the observation predictions, and hi(.) the observation operator mapping the simulated land surface variables to observed quantities. Bias in the observation-minus-forecast residuals is addressed prior to the analysis (Sect. ). The ensemble is created by perturbing the model forcing, the model forecasts, and the observations (Sect. ). The Kalman gain is calculated as Kk,i=Cov(x^k,i-,y^i-)Cov(y^i-,y^i-)+Ri-1, where Cov(x^k,i-,y^i-) is the (sample) error covariance (across the ensemble) between the forecasted land surface state and the forecasted Tb or SM. Similarly, Cov(y^i-,y^i-) is the (sample) error covariance of the Tb or SM forecasts, and Ri is the Tb or SM observation error covariance. The Kalman gain is identical for all ensemble members.

In the case of SM retrieval assimilation, the observation operator hi(.) performs the spatial aggregation of soil moisture simulations from the 36 km grid cells to the satellite footprint; in the case of Tb data assimilation, the observation operator includes both the RTM and the spatial aggregation of gridded Tb simulations to the footprint (Sect. ). For the Tb_7ang assimilation, one observation set at location κ contains Tb observations at a maximum of seven angles and both H- and V-polarization, i.e., up to 14 individual observations yλ,κ,i∈yκ,i. The subscript λ refers to the polarization and incidence angle of the individual Tb observations. In the middle part of the swath, all 14 observations are typically available, whereas slightly fewer observations are available in the outer portions of the swath, where the observations with lower incidence angles are missing.

For the Tb_fit assimilation, one observation set usually contains two observations, i.e., both H- and V-polarization Tb at a 40∘ incidence angle. For the SM retrieval assimilation, each observation set contains only one observation. In all cases, the observation vector yij collects multiple perturbed observation sets that are spatially distributed within an influence radius of 1.25∘ around the model grid cell k, and each observation vector yij has a forecasted counterpart y^ij-. After removal of the persistent errors (Sect.  ) from the O–F residuals (or innovations), the increments Kk,i[yij-y^ij-] are calculated and applied to the state variables. Figure  illustrates the forward simulation from 36 km gridded land surface simulations to footprint-scale observation predictions of Tb and the downscaling of the footprint-scale Tb innovations to 36 km gridded land surface increments.

The subset of prognostic variables updated in Eq. () differs depending on the assimilation experiment. The state vector for Tb assimilation (x= [catdef, srfexc, rzexc, tc1, tc2, tc4, ght1]T) includes prognostic variables related to soil moisture and soil temperature (Sect. ), because Tb observations are by definition sensitive to surface soil moisture and temperature. In contrast, the state vector for SM retrieval assimilation (x= [catdef, srfexc, rzexc]T) contains only model prognostic variables related to soil moisture, because the SM retrievals do not carry direct information about the soil temperature. The selected updates will be propagated to all other variables within the land surface modeling system through energy and water exchange between various soil layers and land–vegetation–atmosphere compartments. For the discussion of the soil moisture increments we will focus on the total profile water increments (Δwtot=Δsrfexc+Δrzexc–Δcatdef) in units of kg m-2 (that is, mm of water equivalent). This quantity is easily understandable and thus simplifies the discussion.

Figures  and  illustrate the concept for Tb assimilation and SM retrieval assimilation, respectively. Figure a–b show swaths of footprint-scale bias-corrected Tb_fit innovations (mapped onto the 36 km EASEv2 grid), for H- and V-polarization at a 40∘ incidence angle from the single-angle Tb assimilation system. The Tb innovations are then transformed into soil moisture and temperature increments using Eq. (). Where Tb innovations are warm, the soil water is reduced and the temperature is increased. Figure c shows the total profile water increments Δwtot and Fig. d shows increments to the first soil layer temperature Δtp1. Increments to the surface temperature prognostic variables (Sect. ; Δtc1, Δtc2, Δtc4) are similar (not shown). Finally, the increments are added to the forecasted fields to create spatially complete analysis maps of surface and root-zone soil moisture, as well as surface temperature and soil temperature (Fig. e–g).

Similarly, Fig. a shows the SM innovations from the SM retrieval assimilation at the same time as in Fig. . Areas with positive (wet) SM innovations in the SM retrieval assimilation roughly correspond to negative (cold) Tb innovations in the Tb assimilation system (Fig. a–b). Note that the color bars for Tb and SM throughout the paper are chosen according to the rule of thumb that a 2–3 K change in Tb corresponds to a 0.01 m3 m-3 change in soil moisture, but keep in mind that the relationship between Tb and SM is nonlinear and varies with time, location, and incidence angle. Next, the SM innovations are converted to soil moisture increments (Δwtot; Fig. b); no increment to surface or soil temperature is calculated. Figures c and b show that the Tb and SM retrieval assimilation systems produce wtot increments with somewhat different large-scale patterns, which is further discussed in Sect. . Finally, Fig. c–d show the resulting surface and root-zone soil moisture analysis fields obtained by adding the increments to the model forecast fields. For both the Tb and SM retrieval assimilation systems, the analysis increments blend smoothly into the forecast fields; that is, the analysis maps do not reveal sharp spatial edges that would reveal the geometry of the assimilated satellite swaths. Further details about this figure are discussed in Sect. .

To limit the long-term biases between Tb observations and simulations, the RTM was calibrated (Sect. ). The 5-year average absolute bias between SMOS Tb and forecasted Tb is about 2 K across the domain. In general, slightly warm model biases are found in the boreal zones and cold model biases over the central part of the US (not shown), but larger seasonal Tb biases remain, primarily due to systematic errors in the modeled temperature and vegetation. The seasonally varying climatological Tb bias is removed prior to data assimilation for each angle, polarization, and overpass time separately, as described in . The Tb innovation biases are calculated over the period 1 July 2010–1 May 2015 for each individual 36 km grid cell without spatial sampling.

The CLSM soil moisture was not calibrated for lack of global observations that would support such an effort and because modeled soil moisture does not necessarily represent soil moisture as observed in the field anyway . Unlike biases in Tb innovations, the biases in the SM innovations are more stationary and do not depend on seasonal temperature variations. Therefore, the SM innovation biases are not corrected seasonally, but instead cumulative distribution function (CDF) matching between the observations and simulations is performed to reconcile the differences in long-term mean, variance, and higher moments, as in earlier retrieval assimilation studies . The observed and simulated SM CDFs are computed for the entire study period, i.e., for 1 July 2010–1 May 2015, at each 36 km grid cell individually.

The imposed ensemble forecast perturbations for Tb and SM retrieval assimilation are identical to those of and not repeated here. The total observation error standard deviation for SMOS Tb_7ang is set to 6 K, which yields near-optimal assimilation diagnostics on average across the globe. However, the diagnostics are not necessarily near-optimal in individual regions . The input observation error standard deviation for SM retrievals is 0.04 m3 m-3, in line with the soil moisture accuracy requirement for the recent SMOS and SMAP missions. The SM retrieval error standard deviation is rescaled following the CDF matching of the SM observations and results in an effective mean error standard deviation of 0.02 m3 m-3, with larger values in the wetter eastern part, which exhibits a higher temporal variability in soil moisture simulations, and lower values in the drier, western part of the study domain (not shown). In all cases, the spatial observation error correlation length is 0.25∘. In the case of multi-angle Tb_7ang assimilation, interangular error correlations are imposed as in .

Observation errors in Tb data or SM retrievals are a combination of instrument error and representation error . The 6 K Tb error consists of a radiometric error of about 4 K for individual incidence angles (instrument error) plus 4.5 K representation inaccuracies (in our system, i.e., based on the near-optimal 6 K observation error) due to errors in the RTM, the spatial aggregation, or other discrepancies between Tb observations and forecasts (6=42+4.52). For Tb_fit observations, the instrument error may be slightly reduced compared to that for Tb_7ang after the angular smoothing, but the representation error remains similar. SM observations contain retrieval errors due to errors in the RTM and in the input L1 Tb observations, as well as representation error due to, e.g., the inherently different nature of simulated and observed soil moisture . In either case, the representation error depends on the soil moisture and temperature dynamics and should ideally be modeled as a function of time and location, but we chose a constant input observation error standard deviation in this paper for simplicity. For SM retrieval assimilation, some spatial error variability is introduced after rescaling in line with the CDF matching.

In our experiments, we do not expect the SMOS Tb and SM retrieval assimilation systems to yield the same results. During the SMOS L2 SM retrieval optimization, the Tb data are used to estimate surface soil moisture and vegetation opacity, given soil temperature background fields provided by the European Center for Medium-Range Weather Forecasts (ECMWF) and look-up parameter information that differs significantly from the NASA GEOS-5 land data assimilation system. In contrast, our SMOS Tb assimilation scheme estimates soil moisture and temperature, given vegetation information. Furthermore, the data screening is necessarily different for Tb data and SM retrievals, and the approach for bias correction is intentionally different. The soil moisture information extracted during the L2 retrieval process or Tb assimilation is thus by design expected to be different. Finally, differences in the Tb and SM retrieval assimilation results could also be due to differences in how close each of the systems is to an optimal calibration of its model and observation error parameters.

The above discussion highlights similarities and stark contrasts in how the Tb and SM retrieval assimilation systems operate. In this section, we look at the effect of these differences on the skill of the assimilation estimates vs. in situ observations. Figure  shows the RMSDub (Sect. ) for the model-only open-loop (OL) simulation, and the change in RMSDub (Sect. ) between the OL simulation and either the Tb_7ang or SM retrieval data assimilation (DA) experiment (ΔRMSDub = RMSDub(DA) – RMSDub(OL)) at individual SCAN and USCRN sites, for the period 1 July 2010–1 May 2015. The gray background shading indicates areas with modest topographic complexity and vegetation cover and where the satellite observations are most sensitive to surface soil moisture details in. The OL simulation has an average RMSDub value of 0.054 m3 m-3 for surface soil moisture and 0.039 m3 m-3 for root-zone soil moisture. Looking more closely, the RMSDub values are generally higher in the central and wetter eastern regions. In dry areas, the RMSDub is limited, because the time series show a limited variability for lack of much precipitation. On average, both assimilation experiments introduce improvements at about 80 % of the sites for surface soil moisture, with spatially averaged ΔRMSDub values of -0.004 and -0.003 m3 m-3 for Tb_7ang and SM retrieval assimilation, respectively. (Spatial average metrics are computed using a cluster-based algorithm, Sect. .) The improvements are also propagated to the root-zone soil moisture (65 % of sites improved) with smaller average ΔRMSDub values of -0.002 and -0.001 m3 m-3, respectively.

The domain-average ΔRMSDub values caused by assimilation are only barely statistically significant for surface soil moisture in “favorable” areas, i.e., where the satellite observations are most sensitive to soil moisture (indicated with green background shading in Fig. ). The differences between Tb_7ang, Tb_fit, or SM retrieval assimilation are not significant. The assimilation contributes an average relative improvement in surface soil moisture of 7 % of the OL RMSDub in favorable locations and 4 % in non-favorable areas. Both Tb and SM retrieval assimilation show improvements in the central and eastern parts of the US, but perform poorly in the western dry mountain areas, where the RMSDub for the OL was small and the assimilation may have introduced some additional noise. The Tb_7ang assimilation shows the largest improvements in the central US, whereas the SM retrieval assimilation shows the largest improvements in the southeastern part, for both surface and root-zone soil moisture. It is possible that the Tb assimilation has a larger impact in the central US than the SM retrieval assimilation, because irrigation events may be filtered in the SM retrievals (and perhaps partly assigned to vegetation opacity retrievals).

The bar plots in Fig.  summarize the average anomR values for the open-loop and data assimilation experiments, after stratifying all SCAN and USCRN sites into “favorable” and “non-favorable” categories (gray vs. white background in Fig. ). The figures show that the open-loop anomR values for surface soil moisture are similar for both the favorable and non-favorable areas (0.51 and 0.50, respectively). However, data assimilation has a larger impact in favorable areas, where all assimilation schemes introduce significant improvements (anomR = 0.63, 0.61, and 0.59 for Tb_7ang, Tb_fit, and SM retrieval assimilation). In non-favorable areas, the improvements are smaller but still significant (anomR = 0.57, 0.56, and 0.54, for Tb_7ang, Tb_fit, and SM retrieval assimilation).

In the root zone, data assimilation also improves the skill over the open-loop simulations, but without statistical significance. The open-loop simulations yield anomR values of 0.56 and 0.50 in favorable and non-favorable areas, respectively. In favorable areas, the assimilation increases the anomR to 0.64, 0.64, and 0.62, for Tb_7ang, Tb_fit, and SM retrieval assimilation. In non-favorable areas, the skill improvement is limited and the anomR values are 0.54, 0.54, and 0.52, for Tb_7ang, Tb_fit, and SM retrieval assimilation. In any case, with assimilation, all anomR values exceed 0.5, meaning that the skill becomes better than a climatological forecast (Brier skill score larger than 0).

Overall, the skill metrics are comparable for the Tb_7ang and Tb_fit assimilation (Fig. ). The results from SM retrieval assimilation are slightly worse than those from Tb assimilation, which may indicate that Tb observations indeed still contain more information (Sect. ) than the SM retrievals, which are implicitly filtered during the retrieval process. However, the differences between the domain-averaged skill values of the various assimilation schemes are minimal. Furthermore, when running the assimilation scheme with different spatially constant Tb observation error parameters, the skill metrics only changed marginally. This shows that our skill metrics are relatively insensitive to uniform changes in the data assimilation parameters. One reason for this is that the skill metrics are presented as (clustered) spatial averages, which compensate for large local differences. It is expected that the skill of our data assimilation systems can only be further improved by using a more localized (in space and time) approach to optimizing the assimilated observations (e.g., L2 SM retrievals) and the forecast and observation error parameters in the EnKF.

Finally, unlike , the skill improvements in this study are smaller when we correct the re-analysis precipitation input with gauge-based precipitation data . This and other recent improvements in the GEOS-5 modeling system make it increasingly challenging to obtain significant skill improvements from the assimilation of microwave observations over areas for which high-quality forcing data are available, such as the domain studied here. The benefits of the microwave-based soil moisture assimilation system are expected to be greater in areas with poorer ancillary inputs to the modeling system. This aspect will be further investigated through the validation of the global SMAP L4_SM data product.