The Australian Water Resources Assessment Landscape (AWRA-L) model (Van Dijk, 2010) underpins the annual national water resource assessments and water use accounts for Australia (Frost et al., 2018; Vogel et al., 2021). The operational implementation of the AWRA-L by the Australian Bureau of Meteorology provides daily 0.05∘ (approximately 5 km) national gridded water balance estimates. The outputs from the operational AWRA-L have been widely used in various agricultural applications and natural resources risk assessment and planning, including commodity forecasting, irrigation scheduling, flood and drought risk analysis, as well as flood forecasting (Frost et al., 2018; Hafeez et al., 2015; Nguyen et al., 2019; Van Dijk et al., 2013; Van Dijk and Renzullo, 2011). The version of the AWRA-L model used in the study was obtained from the Community Modelling system (AWRA-CMS) and is freely available from https://github.com/awracms/awra_cms (last access: 1 September 2020).

AWRA-L is a one-dimensional grid-based model that simulates water balance for each grid cell across the modelling domain by distributing rainfall influx into plant-accessible water, soil moisture and groundwater stores and computing outflux such as evapotranspiration, runoff and deep drainage. The soil water column is partitioned into three layers (surface: 0–10 cm, shallow: 10–100 cm, and deep: 1–6 m) and simulated separately for two hydrological response units, i.e. deep-rooted (trees) and shallow-rooted (grass) vegetation. The water storage for the surface-layer soil is termed S0, while Ss is used for the shallow layer and Sd for the deeper layer. In addition to the modelling of soil columns, the model includes a surface water and a groundwater storage that are simulated at each grid cell and conceptualized as a small unimpaired catchment. In this study, we used forcing inputs from the AWAP (Australian Water Availability Project) gridded climate data, including daily precipitation, air temperature and solar exposure (Jones et al., 2009) and interpolated site-based wind speed (McVicar et al., 2008). It is acknowledged that the accuracy of the model estimates is limited in regions with insufficient coverage in the ground-based observation network (e.g. rain gauges), which is the raw source of AWAP-gridded data used to force the model. This is limited to very remote and mostly uninhabited arid regions in Australia.

To maximize daily spatial coverage, we used two satellite soil moisture products derived from passive L-band systems: the Soil Moisture Active Passive (SMAP) product from NASA (Entekhabi et al., 2010) and the product from the European Space Agency's (ESA's) Soil Moisture and Ocean Salinity (SMOS) mission (Kerr et al., 2001). The SMAP product is the level-2 enhanced radiometer half-orbit 9 km EASE grid soil moisture (Chan et al., 2018). The SMOS product is the level-2 soil moisture product on a ∼25 km grid (Rahmoune et al., 2013). Both SMAP and SMOS produce volumetric soil moisture estimates (in m3 m-3) of approximately the upper 5 cm of soil. Available swath data for each product covering Australia were collated for each 24 h period, approximating the AWRA-CMS operational time steps and resampled to a regular 0.05∘ grid across the modelling domain using bilinear interpolation from 2015 to 2019. The volumetric soil moisture retrievals from both SMAP and SMOS were converted into water storage units (mm) to be consistent in units and soil depths with model estimates, using mean and variance matching to remove the systematic bias. Figure 1 shows an example of daily composites of SMAP (Fig. 1a) and SMOS (Fig. 1b) soil moisture retrievals in model units compared to AWRA-L estimates of S0 (Fig. 1c). For regions with sparse rain-gauge coverage such as central Western Australia (Fig. 1c), AWRA-L modelled S0 persists as zeros or very low values for the experiment period, reflecting a deficiency in the gauge-based analysis of daily rainfall used to drive model simulations. The result of mean and variance matching in these gauge-sparse areas will flatten the variability of SSM time series to zero when using values of the modelled S0 for these areas directly. To resolve this problem and fully leverage the information available in the SSM products to fill the gaps in modelled outputs across the continent, we derived a set of coefficients for the mean and variance matching over the gauge sparse regions by sampling modelled and SSM data from cells surrounding the gaps. Specifically, we fitted a linear model between the maximum SSM values through time and the coefficients for mean and variance matching for each cell in neighbouring region. We applied the derived linear relationship to estimate the corresponding “slope” and “intercept” from the maximum SSM values in the rainfall gaps. This provided a transformation of the SSM into water storage units (mm) and ensures the assimilation can effectively influence the spatial pattern of soil moisture over the sparsely gauged regions.

Triple collocation (TC) was developed as a method of quantifying error characteristics in geophysical variables when the true error structure is elusive. It was first applied to near-surface wind data (Stoffelen, 1998) and later extensively applied to soil moisture (Chen et al., 2018; Crow and Yilmaz, 2014; Dorigo et al., 2017; McColl et al., 2014; Scipal et al., 2008; Yilmaz and Crow, 2014; Zwieback et al., 2013; Crow and Van den Berg, 2010; Su et al., 2014) and rainfall (Alemohammad et al., 2015; Massari et al., 2017). The assumption of this approach is that three independent data sets of the same geophysical variable can be used to infer the error variances in each. Here we use TC as a way of inferring error variances from our three independent estimates of surface soil moisture, AWRA-L S0, SMAP and SMOS from 2015 to 2019. These three collocated measurements were assumed to be linearly related to the true value with additive random errors. To ensure the errors from the three independent sources were unbiased relative to each other, SMAP and SMOS soil moisture retrievals were rescaled to the reference model estimates (AWRA-L S0) using temporal mean and variance matching. McColl et al. (2014) show that the error variances (σ2) of each data set can be calculated from the temporal variance and covariance between data sets respectively as 1σx2=Qx,x-Qx,yQx,zQy,z,σy2=Qy,y-Qx,yQy,zQx,zandσz2=Qz,z-Qz,yQx,zQx,y, where x, y and z denote AWRA-L, SMAP and SMOS soil moisture estimates respectively, and Q denotes temporal variance and covariance between the data sets. These estimates of error variance are used in the determination of the weighting of each data source in the data assimilation (Sect. 3.2).

The data assimilation method used here is a time-sequential updating of model state(s) given observations of relevant model variables (Reichle, 2008). There are two key modelling components in data assimilation: the dynamics operator, which describes the time integration of the system states and fluxes, which in this study is the AWRA-CMS; and the observation operator, which provides the mathematical mapping from state to observation space. The role of the observation operator is to perform a mapping between observation and state space, as often observations are not directly comparable to model states.

The common state updating equation for sequential data assimilation is written as 2xta=xtf+Ktyt-Hxtf, which says that the best estimate of model state, known as analysis (xta), is equal to the first guess or forecast (xtf) plus a weighted difference between observations, yt, and the model equivalent to the observation, H(xtf), for that time step. In this study, the AWRA-L model soil water storage in S0 for shallow-rooted vegetation and deep-rooted vegetation at the surface layer is updated directly through the sequential data assimilation. Satellite surface soil moisture (SSM) products from both SMOS and SMAP are used as the observations to update the model simulation. The observation operator H here is the aggregation of soil water storage estimates in the top-soil layer for two land cover types, i.e. shallow-rooted vegetation and deep-rooted vegetation. When both SMAP and SMOS observations are available, Eq. (2) can be written as a weighted linear combination of model estimates (xtf) and satellite observations (ytSMAP: SMAP observations, ytSMOS: SMOS observations) as 3xta=Kxxtf+KyytSMAP+KzytSMOS. The gain factor, K, contains the error variances (σ2) for both model estimates and observations and can be written as 4Kx=1σx21σx2+1σy2+1σz2Ky=1σy21σx2+1σy2+1σz2andKz=1σz21σx2+1σy2+1σz2, where x, y and z denote the AWRA-L, SMAP and SMOS soil moisture retrievals respectively. If only one satellite observation is available for a time step, the gain factor is calculated using the error variance from the corresponding observation. If neither SMAP nor SMOS are available, the analysis remains the same as the model forecast. Observation error variance is often estimated through field campaigns (Draper et al., 2009; Panciera et al., 2013), but these rarely represent the spatial and temporal variability of errors in gridded satellite products. Alternatively, data providers often specify error estimates, but their magnitude can be overly optimistic. Here, we applied the triple collocation approach (Sect. 3.1) to characterize the temporal error variances of the model estimates and the two satellite observations for each grid cell across Australia. The analysis receives a higher contribution from observations with smaller error variance (Eq. 2). Given the relatively short time series (small number) of observations, however, a single set of error variances is calculated for all time. This results in spatially varying but temporally static error variances (and thus gain weights) for each of the three sources (Fig. 2). We acknowledge the limitations of assuming a temporally constant error variances, and future refinements to the assimilation method will consider introducing seasonally varying error variances.

The assimilation of satellite soil moisture temporarily violates mass conservation in the model through the analysis update. The difference between the analysis, xta, and the forecast, xtf (known as the analysis increment), represents an amount of water that has been added or subtracted from the system that was not present at the start of model integration for the given time step. In this study, we use the concept of tangent linear modelling (Errico, 1997; Giering, 2000) to redistribute the analysis increment of surface soil water storage, S0, to all the relevant model states and fluxes as a way of maintaining mass (i.e. water) balance within each model time step. This adjustment is applied after the sequential state updating as the second step in the assimilation framework, which we refer to as analysis increment redistribution (AIR).

The adjoint and tangent linear models were originally used in variational data assimilation (Bouttier and Courtier, 2002) and have been used to estimate the sensitivity of model outputs with respect to input (Errico, 1997). If we assume the input perturbation here is the analysis increment after the data assimilation (i.e. xta-xtf from Eq. 2), then the change in other model outputs due to the change in inputs can be determined through tangent linear modelling. Assuming model variable b is related to the state variable x, the relationship between them can be simply described as 5b=M(x), where M denotes the model operator. The change in output variable Δb at time step t due to the input change Δx can be determined by 6Δbt=∂M∂xtΔxt. In this study, we applied the tangent linear modelling approach to correct the model forecast of soil water storage for shallow-layer (Ss) and deep-layer soil water storage (Sd), evapotranspiration (Etot) and total streamflow (Qtot) after the state updating of surface soil moisture (S0) at each time step. Note that this process ensures that the correction is affecting all model states in proportion to their sensitivity against changes in the S0. All the model equations regarding the mass redistribution were derived using model equations (Frost et al., 2018; Van Dijk, 2010) and can be found in Appendix A.

Error variances were derived using TC for AWRA-L model estimates and the SSM products and showed that SMAP soil moisture had smaller error variance than SMOS and the model estimates for the majority of the grid cells over the continent. This is consistent with other studies that have shown SMAP provides the best-performing satellite soil moisture product over the majority of applicable global land pixels (Chen et al., 2018). Figure 2 shows the relative weightings (derived from the TC error variances) of model estimates, SMOS and SMAP soil moisture in the data assimilation. The analysed surface soil water storage estimates (S0) receive a greater contribution from SSM products, in particular SMAP observations, compared to model simulations (Fig. 2). Figure 3 gives an example of the temporal change in modelled S0 estimates before and after the assimilation for 2017. The temporal dynamics of S0 estimates after the assimilation has been highly adjusted towards SSM retrievals and in consistency with in situ measurements.

AWRA-L model simulations are driven by gauge-based rainfall analyses. As such the model has difficulty in adequately simulating soil moisture patterns over regions lacking in rain gauge coverage, such as Western Australia and central Australia (Fig. 1c). Water storage simulations over these regions default to zero; thus very little or no weight was given to the AWRA-L estimates in these regions (Fig. 2a). Figure 4 shows different spatial patterns of daily average S0 estimates for December 2019 from model open loop (OL) without data assimilation and with data assimilation through TC-derived weighting (DA-TC). Data assimilation has the effect of adding moisture to AWRA-L S0 simulations over most gauge-sparse areas, as shown in Fig. 4c. Analysed AWRA-L simulations of S0 are dominated by the satellite SSM data as a result of TC weighting in the region, which largely eliminates the erroneous artefacts associated with deficient rainfall data forcing. Reduced water storage in the surface layer of the soil column was found over the south-east of Australia, particularly within the Murray–Darling Basin. This suggests that AWRA-L OL simulations underestimated the severity of the drought experienced in the region in December 2019. The analysis increments of AWRA-L S0(xa-xf) were compared with the difference between in situ rainfall observations from OzFlux network, POzFlux, and AWAP rainfall forcing, PAWAP (Fig. 5). The increasing S0 simulations align with missing or underestimated rainfall events in the AWAP rainfall forcing (POzFlux-PAWAP>0) and vice versa (Fig. 5). This supports the hypothesis that data assimilation correctly distributes water into the system and mitigates the impact of uncertainty in rainfall forcing.

If the analysis increment redistribution (AIR) is not applied, the soil water storage in the surface layer (S0) is the only state variable directly updated with SSM (DA-TC). Other variables such as root-zone soil water storage, evapotranspiration and streamflow are adjusted with model integration to the next time step using the analysed S0 as the surface layer initial condition. Therefore, the observed changes in those variables following DA-TC (Fig. 6, centre column) are relatively small when compared to model open-loop estimates (Fig. 6, left column). For example, the OL soil water storage of shallow-layer (Ss) estimates in those gauge-sparse regions of Australia remain zero or very low due to the AWAP rainfall forcing. The predictions of Ss receive a relatively small contribution from the analysed S0 since the analysis increment of S0 is small compared to the field capacity of Ss.

One known issue of sequential state updating is the temporary break of water balance at each time step until the next model integration. The proposed AIR approach (Sect. 3.2) adjusts variables coupled with surface soil moisture after the state updating at each time step. A significant difference in the spatial patterns of Ss, Etot and Qtot after the mass redistribution (DA-TCAIR) can be seen in Fig. 6 (right column) compared to model open loop or forecasts after only S0 updating. The changes in estimates of Ss and Etot over coastal regions are relatively small due to more accurate rainfall forcing data with the dense network of rain gauges. Finally, the Qtot estimates after AIR are lower than the DA-TC and OL. This reduction in streamflow over south-eastern Australia and northern Australia is consistent with the reduced surface soil moisture in those regions (Fig. 4c).

Estimates of surface soil moisture, root-zone soil moisture, evapotranspiration and streamflow after data assimilation (DA-TC) and data assimilation with mass redistribution (DA-TCAIR) were compared with time series of in situ observations. We compared the model outputs after DA-TC and DA-TCAIR separately to investigate the benefits of maintaining mass balance in data assimilation. Pearson's correlation coefficients were computed from time series of model estimates and observations between January 2016 and December 2018 for each site. The distribution of correlation coefficients for OL, DA-TC and DA-TCAIR is displayed as boxplots in Fig. 7. Consistent, significant improvement in modelled surface layer soil water storage estimates (S0) was observed across all sites (Fig. 7a), with the single exception of an OzFlux site located in a tropical rainforest, where microwave SSM retrievals are known to be typically poor (Njoku and Entekhabi, 1996). TC-based assimilation (DA-TC) increases the correlation between in situ surface SM measurements from 0.47 to 0.72 on average for CosmOz sites, 0.54 to 0.69 for OzFlux sites and 0.56 to 0.77 for OzNet sites compared to OL. This is a significant improvement in AWRA-L simulations of surface soil moisture dynamics, with an increase in correlation of 0.23 on average across all in situ sites.

Overall subtle improvements were observed across the AWRA-L estimates of root-zone soil water storage, evapotranspiration and streamflow after the assimilation (DA-TC) (Fig. 7b–d). This level of improvement is not surprising since those variables were not directly updated through DA-TC and are only influenced through the integration of the model to the next time step. Degradation was found in root-zone soil moisture estimation for a few OzFlux and OzNet monitoring sites. This is likely due to the break of water balance in the assimilation, since the estimates followed by the second step of AIR (DA-TCAIR) slightly increase the correlation with in situ observations compared to model open loop and the estimates after assimilation without mass redistribution (DA-TC). Moreover, the model estimates of root-zone soil moisture from model OL are in good agreement with in situ observations, with an average correlation above 0.8 (Fig. 7b), which leaves little room for improvement.

Although the corrections of other water balance estimates from the analysis increment redistribution are relatively small compared to direct state updating, they are improvements nevertheless. Slight improvements were found similarly in streamflow estimates after the AIR (Fig. 7d). Figure 8 shows an example of the OL estimates of streamflow, the analysed streamflow after the application of AIR and the streamflow observations, Qtotobs. Also displayed is the streamflow analysis increments, i.e. Qtota-Qtotf, for each time step. The negative streamflow analysis increment (Fig. 8) indicates that water is removed from the surface water store after the assimilation of SSM and application of AIR, which appears to compensate for the overall overestimate of OL simulations, in this example. Although the change in streamflow due to the soil moisture data assimilation is small compared to the disparity between model and observed streamflow, the adjustment in the direction towards observations highlights the importance of accurate antecedent soil moisture conditions in the simulation of runoff response. The joint assimilation of gauge-measured streamflow and satellite soil moisture retrievals into AWRA-L is expected to improve the streamflow simulation.

A limited number of root-zone soil moisture monitoring sites as well as the large spatial disparity between the point-scale in situ measurements and modelling resolution (∼5 km grid cell) represent substantial limitations for wide-area evaluation of root-zone soil moisture estimates. An indirect verification of AWRA-L simulations of root-zone soil moisture was based on a comparison against satellite-derived EVI. This provided an independent, albeit indirect, way of evaluating the impact of data assimilation over larger areas. We calculated the correlation between time series of monthly average AWRA-L root-zone soil moisture estimates from OL, DA-TC and DA-TCAIR against EVI for cropland across Australia from 2015 to 2018. Cropland cover type was selected based on the rooting depths of the dominant grass species and wheat varieties in the area that have been shown to have rooting depths spanning at least half the combined soil depths (0–1 m) of the surface- and shallow-layer soil water storage in AWRA-L. Figure 9a shows the relative change in correlation between root-zone soil water storage simulations from DA-TCAIR and those from model OL against EVI data for cropland areas of Australia. Significant improvements were found after the data assimilation and mass redistribution for the vast majority of model grid cells (Fig. 9a). The averaged correlation with EVI is 0.64 from DA-TCAIR compared to 0.52 for model open loop. The root-zone soil water storage estimates after the mass redistribution are significantly improved over the cropland in Western Australia and southern Australia, with more than 20 % increase in correlation comparing to DA-TC without mass redistribution (Fig. 9b). This demonstrates that enforcing mass balances as part of the soil moisture data assimilation at each time step is essential to improving the simulation of root-zone soil water balance. Limited differences between DA-TC and DA-TCAIR were found over cropland regions over south-eastern Australia, likely due to the overall good performance of AWRA-L OL root-zone soil moisture estimates in those areas (Fig. 7b). The improved consistency with EVI after data assimilation highlights the potential of improving agricultural planning with more accurate information of root-zone soil water availability.

To quantify how long improvements in model state last in AWRA-L simulations, we used OL and DA-TCAIR estimates between 1 March 2018 and 28 February 2019. The model states for each day over this 1-year period served as initial conditions for 100 d AWRA-L simulations, from which we calculated the number of days it took for the simulation from the analysed DA-TCAIR states to converge to within ±5 % of those from OL. Results showed that data assimilation can impact model states and fluxes for weeks and sometimes up to 2–3 months (Fig. 10). The impacts of data assimilation can persist in simulated S0 for as long as a week over coastal regions and longer in central Western Australia and northern Australia, with up to a month persistence in winter and spring (Fig. 10a). There is less impact on S0 simulations during wet season (summer–autumn) in northern Australia since the S0 can saturate rapidly due to the heavy rainfall. Overall, the longest persistence is found in winter, with a continental average of 13 d; the shortest is 6 d on average in autumn and summer. The memory of initial conditions in simulations of Ss can persist even longer due to the slower response to rainfall variability and higher field capacity (Fig. 10b). Summer persistence for Ss is the least, with a continental average of 30 d; in winter this increased to 45 d.

On average, the impact of antecedent soil moisture conditions on evapotranspiration simulations can persist for 1 week over coastal areas but for up to months in central Western Australia (Fig. 10c). The continental average varies from 13 to 20 d for each season. The areas with the longest persistence are those areas with artefacts of zero rainfall in the forcing. This demonstrates that improvements in AWRA-L estimates after SSM assimilation over regions with sparse rain-gauge coverage can persist in the system for more than 2 months. The impact on runoff varies from 1 week to 3 months over the continent (Fig. 10d). The majority of areas impacted for more than 2 months are in locations of low rainfall and runoff. However, in areas of heavy runoff, e.g. north-eastern Australia, there is between 1–2 weeks of persistence.