The probability distributed model (PDM) is a conceptual rainfall-runoff model that has been widely used in hydrologic research and applications , mainly over temperate and humid environments. The model was selected from amongst the set of models available within the flood forecasting system managed by the Australian Bureau of Meteorology. This selection was based on both the suitability of PDM to simulate ephemeral rivers and preliminary analysis comparing PDM against other models such as the Sacramento soil moisture accounting model, which did not perform as well as PDM.

PDM is a parsimonious model where the runoff production is controlled by the absorption capacity of the soil (including canopy and surface detention). This process is conceptualised by a store with a distribution of capacities across the catchment and the spatial distribution of these capacities is described by a probability distribution . The spatial variability of store capacities can be related to different soil depths, which was identified as the most dominant factor governing runoff variability in a semi-arid catchment .

In the current formulation, the model treats soil moisture store (S1 in Fig. ) over the entire catchment as a distributed variable with capacities (c) following a Pareto distribution function, F(c). At a given time, the different stores receive water from rainfall and lose water by evaporation and groundwater recharge (drainage). The shallower stores with less capacity than a critical capacity, C*, start to generate direct runoff while the rest accumulates the water as soil moisture. The proportion of the catchment that generates runoff can therefore be expressed in terms of the Pareto density function, f(c), as probc≤C*=FC*=∫0C*f(c)dc. In this way, for a time t, the soil moisture over the entire catchment, θ (water content of S1), can be expressed as the summation of all the store capacities greater than C*(t): θ(t)=∫0C*(t)1-F(c)dc. Note that the critical capacity C* varies in a time interval Δt based on the net rainfall rate during that time, P, C*(t+Δt)=C*(t)+PΔt.

Direct runoff is calculated based on Eq. () and routed through two cascade of reservoirs (S21 and S22 in Fig. , with time constants k1 and k2, respectively). Subsurface runoff is estimated based on the drainage from S1 and transformed into baseflow by using a storage reservoir (S3 in Fig.  with time constant kb). These are then combined as total runoff, or streamflow. A detailed description of the model conceptualisation and the formulation of the different rainfall-runoff processes is presented in .

PDM was set up using both a lumped scheme and a semi-distributed scheme (see Fig. ). The semi-distributed scheme was configured with seven sub-catchments (SC1–SC7), each using the lumped version of PDM. The area and mean annual rainfall of each sub-catchment are summarised in Table . The river routing between upstream and downstream sub-catchments in the semi-distributed scheme was represented by a linear Muskingum method : S=kmIx+(1-x)O, where S is the storage within the routing reach, km is the storage time constant, I and O are the streamflow at the beginning and end of the reach, respectively, and x is a weighting factor parameter. The time constant parameters of the storages S21, S22 and S3 (k1, k2 and kb, respectively) were scaled by the area of each sub-catchment, and km from the Muskingum routing was scaled by the length of the river channel between corresponding nodes. The remaining model and routing parameters of the semi-distributed scheme were treated as homogeneous.

The lumped and the semi-distributed models were calibrated by using a genetic algorithm with an objective function based on the Nash–Sutcliffe model efficiency (NSE) . The models were calibrated for the period 1 January 1967–31 May 2003 and evaluation performed for the period 1 June 2003–2 March 2014. To make fair comparisons between the two model setups in a scenario where the inner catchments are ungauged, the semi-distributed scheme was calibrated using only the outlet gauge (N7 in Fig. ). The performance of the calibrated models was evaluated based on the NSE at the catchment outlet (N7, Fig. ) and at inner nodes N1 and N3, in the case of the semi-distributed scheme.

To analyse the runoff mechanisms simulated by the lumped and the semi-distributed schemes, we calculated the lag-correlation between rainfall and streamflow, and between antecedent SM and streamflow. This enables further understanding of the improvement in streamflow that can be expected by improving the simulated SM content through SM-DA.

The EnKF proposed by has been widely used in hydrologic applications given the nonlinear nature of runoff processes. In the EnKF, the error covariance between the model and observations is calculated from Monte Carlo-based ensemble realisations. In this way, the model and observation uncertainties are propagated and the streamflow prediction is treated as an ensemble of equally likely realisations. The uncertainty of the streamflow prediction can be derived from the ensemble, which provides valuable information for operational flood alert systems.

In a state-updating assimilation approach, the state ensemble is created by perturbing forcing data, parameters and/or states of the model with unbiased errors. As we will see in Sect. , an N-member ensemble of model soil moisture, θ={θ1,θ2,…θN}, was generated by perturbing rainfall forcing data, the model parameter k1, and θ. Then, the soil water error of member i at time t was estimated as θi-(t)′=θi-(t)-1N∑i=1Nθi-(t), where the superscript “-” denotes the state prediction prior to the assimilation step. The error vector for time step t was defined as θ-(t)′={θ1-(t)′,θ2-(t)′,…,θN-(t)′} and the error covariance of the model state (P-) was estimated at each time step as: P-(t)=1N-1θ-(t)′⋅θ-(t)′T.

When a daily SM observation from AMS, ASC or SMO was available, each member of the background prediction (θ-) was updated. Before being assimilated, each of the three observation data sets was transformed to represent a profile SM and then rescaled to remove systematic differences between the model and the transformed observations (details in Sects.  and ). We sequentially assimilated an N-member ensemble of the transformed and rescaled AMS, ASC and SMO (named θams,θasc and θsmo, respectively) and updated each member of θ- with the following three steps:

If θams was available at time t,θi+(t)=θi-(t)+K1(t)⋅(θiams(t)-Hθi-(t)),where H is an operator that transforms the model state to the measurement space. Since the additive and multiplicative biases between the model predictions and the microwave retrievals were removed via rescaling in a separate step (see Sect. 3.6), H reduced to a unit matrix. The Kalman gain K1(t) was calculated asK1(t)=P-(t)HTHP-(t)HT+R1(t),where R1(t) is the error variance of θams estimated in the rescaling procedure (Sect. ). If θams was not available, θ+(t)=θ-(t).

In the case of the semi-distributed scheme, during the updating steps described above, each sub-catchment was treated independently and no spatial cross-correlation in the satellite measurements was considered. The order of the products assimilated in steps 1–3 was arbitrary; however, we checked that different orders did not significantly affect the SM-DA results.

The main sources of uncertainty in hydrologic models are the errors in the forcing data, the model structure and the incorrect specification of model parameters . Generally, these errors are represented by adding unbiased synthetic noise to forcing variables, model state variables and/or model parameters.

The estimation of model errors is among the most crucial challenges in data assimilation, as it determines the value of the Kalman gain. In the case of a state updating SM-DA, the ability of the scheme to improve streamflow prediction will mainly depend on the covariance between the errors in SM states and modelled streamflow, which directly depends on the specific representation and estimation of the model errors.

To represent the forcing uncertainty, we adopted a multiplicative error model for the rainfall data . In particular, we followed the scheme used in various SM-DA studies (e.g. ) and represented a spatially homogeneous rainfall error (ϵp) as ϵp∼ln⁡N(1,σp2), where σp is the standard deviation of the log-normal distribution. The above representation assumes a spatially homogeneous fraction of the error to the rainfall intensity, which could be an over-simplification in a large area like the study catchment. However, it avoids the estimation of additional error parameters (e.g. spatial correlation parameter) in an already highly undetermined problem (see Sect. ).

The parameter uncertainty was represented by perturbing the time constant parameter (k1) for store S21, a highly sensitive parameter of the model that directly affects the streamflow generation by influencing the water stored in both surface storages S21 and S22 (note that in the PDM formulation used, the time constant k2 is calculated as a function of k1). Given the lack of prior information about the structure of the parameter error (ϵk), we adopted a normally distributed multiplicative error with unit mean and standard deviation of σk, following previous SM-DA studies working with rainfall-runoff models .

Following the scheme used in most SM-DA experiments (e.g. ), the model structural error was represented by perturbing the SM prediction (θ) with a spatially homogeneous additive random error, ϵs∼N(0,σs2), where σs is the standard deviation of the normal distribution.

The physical limits of SM (porosity as an upper bound and residual water content as a lower bound) are represented by the model through the storage capacity of S1. When θ approaches the limits of S1, applying unbiased perturbation to θ can lead to truncation bias in the background prediction. This can then result in mass balance errors and degrade the performance of the EnKF . Moreover, the Kalman filter assumes unbiased state variables. This issue is of particular importance in arid regions like the study area, where the soil water content can be rapidly depleted by evapotranspiration and transmission losses, thus approaching the residual water content of the soil. To ensure that the state ensemble remained unbiased after perturbation we implemented the bias correction scheme proposed by .

The truncation bias correction consisted of running a single unperturbed model prediction (θ-0) in parallel with the perturbed model prediction (θi,-). At each time step, the mean bias, δ(t), of the N-member ensemble prediction was calculated by subtracting θ-0(t) from the ensemble mean, as follows : δ(t)=1N∑i=1Nθi-(t)-θ-0(t). Then, a bias corrected ensemble of state variables, θ̃i-(t), was obtained by subtracting δ(t) from each member of the perturbed ensemble, θi-(t).

Although the latter resulted in unbiased state ensembles, some important but subtle effects remain that arise from the highly non-linear nature of hydrologic model. These need to be guarded against in SM-DA. Representing model errors by adding unbiased perturbation to forcing, model parameters and/or model states can lead to a biased streamflow ensemble prediction (e.g. ), compared with the unperturbed model run. This biased streamflow ensemble prediction (open-loop hereafter) is degraded compared with the streamflow predicted by the unperturbed calibrated model. As a consequence, improvement of the open-loop after SM-DA will in part be due to the correction of bias introduced during the assimilation process itself.

To avoid overstating the SM-DA efficacy due to the above issue, we applied the bias correction scheme directly to the streamflow prediction (in both the open-loop and the assimilation runs). We used the unperturbed model run to estimate a mean bias in the streamflow (following Eq. , but using streamflow instead of soil moisture) and then corrected each ensemble member by subtracting this mean bias. This practical tool ensures that the streamflow ensemble mean maintains the performance skill of the unperturbed (calibrated) model run, thus avoiding artificial degradation of the unperturbed model run by bias. To our knowledge, this approach has not been applied in previous SM-DA studies.

To calibrate the error model parameters (σp, σk and σs), we evaluated the open-loop ensemble prediction (Qol) against the observed streamflow at the catchment outlet. In this study we used a maximum a posteriori (MAP) scheme, a Bayesian inference procedure detailed by that maximises the probability of observing historical events given the model and error parameters. In other words, it maximises the probability of having the streamflow observation within the open-loop streamflow.

Member i from the N-member open-loop can be expressed as Qiol(t)=QT(t)+ϵm(t), where QT is the (unknown) truth streamflow and ϵm is the error of the streamflow prediction and consists of forcing, parameter and states errors: ϵm(t)=f(ϵp(t),ϵk(t),ϵs(t)).

The observed streamflow at N7 (Qobs) can be expressed as a function of the same (unknown) truth and the streamflow observation error (ϵobs), Qobs(t)=QT(t)+ϵobs(t). Combining Eqs. () and (), the model ensemble prediction of the observed streamflow (Q^obs) is expressed as: Q^obs(t)=Qol(t)+ϵm(t)+ϵobs(t).

Following , ϵobs was assumed to be a serially independent multiplicative error following a normal distribution (mean 1 and standard deviation of 0.2). Then, the likelihood function (L) defining the probability of observing the historical streamflow data given the calibrated model parameters (x), and the error model parameters (σp, σk and σs), was expressed as L(Qobs|x,σp,σk,σs)=Πt=1np(Qobs(t)|Q^obs(t)).

To maximise L, we applied a logarithm transformation to it and maximised the sum over time of the transformed function. The probability density function (p) at each time step was estimated by assuming that the ensemble prediction of the observed streamflow, Q^obs(t), follows a Gaussian distribution, with its mean and standard deviation computed using the ensemble members. The period used to calibrate the error model parameters was 1 January 1998–31 May 2003.

An important aspect to highlight about this error parameter calibration is that it is a highly underdetermined problem. Only one data set (streamflow at N7) is used to calibrate the error parameters, while there might be many combinations of error parameters that can generate similar streamflow ensemble (equifinality on the error parameters).

The aim of the stochastic assimilation detailed in Sect.  is to correct θ, which is a profile average SM representing a soil layer depth determined by calibration. By assuming a porosity of 0.46, (A-horizon information reported in ), and the model S1 storage capacity of 396 mm (420 mm) for the lumped (semi-distributed) scheme, this profile SM roughly represents the upper 1 m of the soil. On the other hand, the satellite SM observations represent only the few top centimetres of the soil column (see Sect. ). To provide the model with information about more realistic dynamics of θ, we applied the exponential filter proposed by to the satellite SM to estimate the soil wetness index (SWI) of the root-zone. SWI has been widely used to represent deeper layer SM based on satellite observations (e.g. ). SWI was recursively calculated as: SWI(t)=SWI(t-1)+G(t)SSM(t)-SWI(t-1), where SSM(t) is the satellite SM observation and G(t) is a gain term varying between 0 and 1 as: G(t)=G(t-1)G(t-1)+e-t-(t-1)T. T is a calibrated parameter that implicitly accounts for several physical parameters . T was calibrated by maximising the correlation between SWI and the unperturbed model soil moisture (θ) during the first year of satellite data. This calibration period was selected to maximise the independent evaluation period (see Sect. ); however, more representative values are likely to be obtained if a longer period is used for calibration. SWI was calculated independently for each of the AMS, ASC and SMO data sets (named SWIAMS, SWIASC and SWISMO, respectively) and then rescaled to remove systematic differences with the model prediction (Sect. ).

The systematic differences (e.g. biases) between θ and the SWI derived from each satellite product must be removed prior to applying a bias-blind data assimilation scheme . We applied instrumental variable (IV) regression to resolve the biases and estimate the measurement errors simultaneously . In three-data IV regression analysis, also known as triple collocation (TC) analysis , the model θ, the passive SWI and active SWI are used as the data triplet. As the sample size requirement for TC is stringent , a pragmatic threshold of 100 triplet sample was imposed . During periods when only one satellite product was available (i.e. before ASC) or when the sample threshold for TC was not met, a two-data set IV regression using lagged variables (LV) was applied as a practical substitute . The LV analysis was performed on the model θ and a single satellite SWI, with the lagged variable coming from the model.

In most SM-DA experiments, the error in satellite SM has been treated as time-invariant (e.g. ); however, studies evaluating satellite SM products have shown an important temporal variability in the measurement errors . Since a data assimilation scheme explicitly updates the model prediction based on the relative weights of the model and the observation errors, assuming a constant observation error may lead to over-correction of the model state if the actual error is higher, and vice versa.

Temporal characterisation of the observation error can be achieved by applying TC (or LV) to specific time windows of the observations and model predictions (for example, by grouping the triplets or doublets by month-of-the-year). There is however, a trade-off between the sampling window (which defines the temporal characterisation of the error) and the sample size (number of triplets in each subset). In an operational context this trade-off becomes more critical since only past observations are available. After analysing the temporal variability of the observation errors using the complete period of record (not shown here), we found that a 4-month sampling window can reproduce seasonality in errors while ensuring sufficient data samples for the TC and LV schemes. With this analysis we also assessed the suitability of using LV, which yielded similar results to TC although some positive bias in LV error variance estimates relative to TC was noted (not shown here).

Summarising, the procedure for rescaling and error estimation consists of:

From the start of the AMS data set, we grouped LV triplets (SWIAMS(t), θ(t) and θ(t-1)) into three subsets: December–March, April–July and August–November.

To evaluate the SM-DA results, we used six different metrics. Firstly, the normalised root mean squared difference (NRMSE) was calculated as the ratio of the root mean square error (RMSE) between the updated streamflow ensemble (Qup) and the observed streamflow to the RMSE between the open-loop (ensemble streamflow prediction without assimilation, Qol) and the observed discharge: NRMSE=1N∑i=1N∑t=1TQiup(t)-Qobs(t)21N∑i=1N∑t=1TQiol(t)-Qobs(t)2, where N=1000 is the number of ensemble members. The NRMSE provides information about both the spread of the ensemble and the performance the ensemble mean, which is considered as the best estimate of the ensemble prediction. Moreover, as it is calculated in linear streamflow space, it gives more weight to high flows.

To further evaluate the performance of the ensemble mean, we calculated the Nash–Sutcliffe efficiency (NSE) for the entire evaluation period as follows (example for the open-loop case): NSEol=1-∑tQobs(t)-Qol‾(t)2∑tQobs(t)-Qobs‾2, where Qol‾ is the open-loop ensemble mean. Similarly, NSEup was calculated by applying Eq. () to the updated ensemble mean (Qup‾).

We also estimated the probability of detection (POD) of daily flow rates (not flood events) exceeding minor, moderate and major floods, for the open-loop and the updated ensemble mean, as follows (example for the open-loop case): PODol=#(Qol‾>=Qobs15.7%&Qobs>=Qobs15.7%)#(Qobs>=Qobs15.7%), where the symbol # represents the number of times. Qobs15.7% is the observed streamflow corresponding to a minor flood classification. This corresponds to a flow (not flood) frequency of 15.7 % (see Sect. ). Similarly, PODup was calculated by applying Eq. () to the updated ensemble mean (Qup‾). We estimated the false alarm ratio (FAR) for daily flows as (example for the open-loop case): FARol=#(Qol‾>=Qobs15.7%&Qobs<Qobs15.7%)#(Qobs<Qobs15.7%). Similarly, FARup was calculated by applying Eq. () to the updated ensemble mean.

Finally, we calculated the aggregated peak volume error (PVE, in mm) of the ensemble mean, for days when the observed streamflow was above a minor flood classification (t* days in Eq. ). An example for the open-loop, PVE was calculated as PVEol=∑t*Qol‾(t*)-Qobs(t*).

To evaluate the skill of the streamflow ensemble prediction before and after SM-DA, we calculated the continuous ranked probability score (CRPS; ). CRPS is used as a measure of the ensemble errors. In the case of the deterministic unperturbed run, CRPS reduces to the mean absolute error. The reliability of the ensembles was also evaluated by inspecting the rank histograms of the ensemble following . A reliable ensemble should have a uniform histogram while a u-shape (n-shape) histogram indicates that the ensemble spread is too small (large) .

The evaluation period for the SM-DA was 1 June 2003–2 March 2014. This period is independent of all scheme component calibration periods (see Sects. , and ).

The streamflow at the outlet of the study catchment (N7 in Fig. ) features long periods of zero-flow, a negligible baseflow component and sharp flow peaks after rainfall events, when the catchment has reached a threshold level of wetness (see observed streamflow in Fig. ).

The simulated streamflows from the lumped and the semi-distributed schemes are presented in Fig. . To help visualisation of these time series, the calibration and evaluation periods were plotted separately. The evaluation period was further separated into two sub-periods, evaluation sub-period 1 (1 June 2003–30 April 2007), characterised by having only moderate and minor floods, and evaluation sub-period 2 (30 April 2007–2 March 2014), which had three major flooding events. The plots show that both the lumped and the semi-distributed models are generally able to capture the hydrologic behaviour of the catchment. As expected, the spatial distribution of forcing data and the channel routing accounted for by the semi-distributed scheme enhanced the overall performance of the model, with lower residual values through time (panels a.2, b.2 and c.2 in Fig. ) and consistently improved the simulation of peak flows.

Table  presents the evaluation statistics for the streamflow prediction in the calibration and evaluation periods, for both the catchment outlet and the inner catchments (notice that N1 does not have data in the calibration period). The different statistics in this Table consistently show that, at the catchment outlet, the semi-distributed has consistently better performance than the lumped scheme in terms of RMSE, NSE, PVE and CRPS. Both schemes show better statistics in the evaluation period due to the higher flows over that period.

The good performance of the semi-distributed scheme at the catchment outlet was not reflected at the inner catchments. To explore the reasons for such bad performance, we separately calibrated the model parameters in those sub-catchments by using all the available N7, N1 and N3 observations. The results (not shown here) revealed that in this case, the model was able to adequately simulate streamflow in those sub-catchments (NSE in evaluation period of 0.78, 0.69 and 0.84 at N1, N3 and N7 nodes, respectively). Based on this, we argue that the problem of the poor model performance in the “ungauged” inner catchments is most likely due to sub-optimal parameter estimation (due to the limited information about catchment heterogeneity provided by the integrated catchment streamflow response) and unlikely to be due to errors in the input data or model structure.

To focus the analysis of the catchment runoff mechanisms on periods with flood events, the lag-correlation between the daily streamflow simulated at N7 and θ (Fig. ), and between daily streamflow and the daily rainfall (Fig. ), was calculated for daily streamflow values greater than Qobs15.7%, or minor flood level. The lumped scheme indicates a stronger link between θ and streamflow than the semi-distributed scheme. This is represented by higher r values in panel (a) compared with panels (b)–(h) in Fig. . Conversely the link between rainfall and streamflow is weaker in the lumped scheme (lower r values in panel (a) compared with panels (b)–(h) in Fig. ). These different representations of the catchment runoff response will have a direct impact on the skill of SM-DA to improve streamflow prediction. A strong relationship between θ and streamflow prediction suggests strong correlation between their errors, and therefore, greater potential improvement of streamflow resulting from an improved representation of θ.

If we assume that the semi-distributed scheme provides a better representation of runoff response within the entire catchment (based on its better model performance at the outlet), Figs.  and also suggest that daily rainfall is the main control on runoff generation and thus has a stronger impact on the streamflow prediction than soil moisture. Figure  shows that flood prediction strongly depends on antecedent soil moisture for up to the preceding 3 days. The strong correlation found at lag-0 suggests that the real time SM correction given by the proposed SM-DA would be a good strategy to improve flood prediction.

The calibrated error parameters for the lumped and the semi-distributed schemes are σp=1.286 mm and 0.977 mm; σs=0.099 and 0.03 and σk=0.084 and 0.018, respectively. σs is expressed as a percentage of the total storage capacity (396 mm in the lumped scheme and 420 mm in the semi-distributed scheme) and σk is expressed as a percentage of the calibrated parameter k1.

The rank histograms of the generated ensemble prediction (open-loop) are presented in Fig. . The histograms at the catchment outlet (N7) are either n-shaped or displaced to one side, for both the lumped and semi-distributed model schemes (Fig. a and b, respectively). This suggests that the open-loop ensembles are slightly biased (with respect to the observed streamflow) and feature wider spread than an ideal ensemble. The width of the spread will be critical for the evaluation of SM-DA (Sect. ) since any decrease of the spread would be considered as an improvement of the ensemble prediction.

The wider spread of the open-loop ensembles at the catchment outlet could be due to factors such as an over-prediction of error parameters by the MAP calibration algorithm, or the representation of the model error with time-constant error parameters. The latter becomes critical given the distinct behaviour of the intermittent streamflow response within the catchment, which could indicate distinct behaviour in the model errors as well.

The ensemble predictions at the inner nodes N1 and N3 (Fig. c and d, respectively) feature high bias with respect to the observed streamflow (note that observations at N1 and N3 were not used to calibrate the error parameters). The large bias at these inner nodes result from the large errors in the calibrated model in SC1 and SC3 (see Sect. ).

The satellite SM derived from AMS, ASC and SMO are presented in Fig. a, for the lumped model. The satellite data sets feature significantly higher noise than the modelled θ. This can be explained by factors such as random errors in the satellite retrievals , and the rapid variation of water content in the surface layer of soil due to infiltration and evapotranspiration losses. Figure b presents the SWI derived from the satellite products, after seasonal rescaling (θams, θasc and θsmo). This plot shows better agreement between model and observations due to SWI filtering/transformation, even when the higher noise in the rescaled SWI time series is still present.

Figure c shows the seasonal observation error variance, and reveals a clear variation in the error with time. The variation of the seasonal error values is due to the alternative use of TC or LV and to the increasing sample size of each seasonal pool (see Sect. ), which should reduce the uncertainties coming from finite sample size. One limitation of this procedure is its assumption that the errors vary seasonally without inter-annual variability. Since there are inter-annual cycles (wet and dry years), one may also expect the errors to vary with year. Ideally, moving-window estimation with windows smaller than 3 months should be considered, but that would cause greater sampling uncertainties for the TC and LV estimates. The inverse relationships between θams and θasc error variances at some times could be due to the passive retrieval by AMS compared with the active ASC, among other factors.

A common error standard deviation value used in previous SM-DA studies is 3 % m3 m-3 (e.g. ). This constant error, when transformed according to the soil moisture storage capacity of the model and the soil porosity (see Sect. ) gives an error variance of 667 (750) mm2 for the lumped (semi-distributed) scheme. As a simple comparison, these values are within the range of the error variance estimated through seasonal LV/TC; however, a comprehensive analysis of the impacts of accounting for seasonality in SM-DA is beyond the scope of this work.

Table  summarises the results of the SWI calibration and seasonal rescaling for the lumped model, showing the T parameter for each SWI and the correlation coefficient (r) between θ and the satellite SM before and after SWI transformation and rescaling (θobs). These results confirm the visual assessment of plots in Fig.  by showing an important increase in the linear correlation coefficient with θ when satellite SM is transformed into SWI. The correlation is further increased after rescaling, which illustrates that there is clear benefit from performing seasonal bias correction. Note that applying a constant rescaling factor would have no impact on the correlation between θ and θobs.

The optimal T values (Table ) are difficult to validate since there is no ground data to compare with and, given that it has been shown that they strongly depend on the physical processes of the study site , direct comparison with other studies cannot be made reliably. Indeed, previous studies have shown a wide range of optimal T values for soil depths ranging between 10 and 100 cm. As an example, in Fig.  we have summarised the optimal T found in five different studies .

Previous studies have shown that optimal T value increases with layer depth (e.g. ). Results presented here show an increased T value for SMO, which would be inconsistent with L-band having a deeper penetration than AMS C-band (to limit the comparison within passive retrievals). We speculate that these differences might be due various factors, including the different retrieval methods (which have quite different assumptions pertaining to spatial heterogeneity) and the influence that radio-frequency interference noise. Moreover, to the best of our knowledge, the existing studies examining the dependence of T on the soil depths are usually based on a single satellite product against in situ measurements at variable depths. Hence it is difficult to compare our results against these studies due to the increased complexity due to different sensing and retrieval methods.

There are some key theoretical issues that should be considered when using SWI as a profile SM estimator. Firstly, the parameter T in Eq. () was estimated by maximising the correlation between SWI and θ, which could introduce cross-correlated errors between them. This would violate the IV regression assumption of no correlation between the errors among the triplets (Sect. ). A way to overcome this issue, if data requirements are met, would be to estimate a profile SM independently of the rainfall-runoff model prediction, for example by using a physically based model to transfer surface SM into deeper layers (e.g. ).

Secondly, the SWI formulation explicitly incorporates autocorrelation terms, which would result in autocorrelated errors in the observation, which violates an EnKF assumption: independence between observation and prediction errors. The autocorrelation in the observation error can be transferred to the updated θ+ during the SM-DA updating step. In that case, the θ- background prediction error covariance at time t+1 would be correlated to the error of the rescaled SWI at time t+1. In contrast with the first issue listed above, the violation of the EnKF assumption can not be avoided by replacing SWI with a physically based model, since the latter would result in profile SM strongly correlated with previous states as well. Indeed, given the physical mechanisms of water flux in the unsaturated soil, this problem will be present whenever a profile SM estimated from satellite SM is used as an observation in an EnKF-based data assimilation framework. A way to overcome this could be to work with models that explicitly account for the water in the top few centimetres of soil and therefore can directly assimilate a (rescaled) satellite retrieval. However, the errors in satellite SM retrievals are probably already autocorrelated .

Breaching some of the EnKF-based scheme and/or the IV-based rescaling assumptions could theoretically degrade the performance of the SM-DA scheme, when the variable analysed is soil moisture . In this context, the performance of SM-DA with respect to the improvement in streamflow has been under-investigated. show that in terms of streamflow prediction, SM-DA seems to be less sensitive to violation of these assumptions. Both the lower sensitivity and the apparent contradiction with previous studies analysing soil moisture prediction performance highlight the need for further studies focusing on SM-DA for the purposes of improving streamflow prediction from rainfall-runoff models.

The ensemble predictions of streamflow and θ, before and after SM-DA, for both the lumped and the semi-distributed schemes at N7, are presented in Fig. . The truncation bias correction (Sect. ) was successful in creating an unbiased θ ensemble when the unperturbed model approached the soil water storage bounds (Fig. a.2 and b.2).

The rank histograms at N7, N1 and N3 are presented in Fig. 7. For all the evaluated nodes, the ensemble predictions are more reliable after SM-DA (flatter histograms compared with the open-loop). The consistent overestimation of the observed streamflow in the open-loop ensembles (diagonal histograms displaced towards the higher ensemble percentiles) is partially addressed by the SM-DA.

The evaluation statistics for the SM-DA are summarised in Table . The streamflow data of the inner catchments (N1 and N3) are used only for evaluation purposes in the semi-distributed scheme, therefore they are representative of “ungauged” inner catchments.

The NRMSE in Table  (all values below 1) demonstrates that the SM-DA was effective in reducing the streamflow prediction uncertainty (RMSE) across all gauged and ungauged catchments. The reductions in the RMSE ranged from 17 to 24 % for the different evaluation nodes. The NRMSE combines precision improvement (i.e. reduction of ensemble spread) with prediction accuracy improvement (i.e. enhancement of ensemble mean performance) resulting from the SM-DA. Given that the ensemble open-loop spread was larger than an ideal ensemble (based on the n-shaped rank histograms in Fig. ), the reduction of the ensemble spread may be in part artificial.

The performance of the ensemble mean was assessed by computing the NSEol and NSEup (Table ). At the catchment outlet, the NSE of the ensemble mean after SM-DA only improved for the semi-distributed scheme. At the ungauged catchments, SM-DA was effective at improving the performance of the ensemble mean only at N3, compared with the open-loop. However, the performance of the model in that catchment was still poor. This can be explained by the systematic errors present in the model for those catchments before assimilation, which were not addressed by the SM-DA.

The POD values at the catchment outlet (N7) show that before and after SM-DA, the model is consistently capable of detecting minor floods. Although this does not demonstrate an advantage of the SM-DA scheme proposed here, it does reflect the adequacy of the model ensemble prediction for simulating minor (and larger) floods. Consistently with previous results, the prediction of the semi-distributed model at the inner catchments is poorer in terms of detecting minor floods. The lower FAR values after SM-DA demonstrates the efficacy of the scheme in reducing the number of times the model predicted an unobserved minor flood, at both the gauged and the ungauged catchments.

The open-loop PVE was improved (lower PVE values) after SM-DA at N7 (for both the lumped and the semi-distributed schemes) and at N3. This was not the case however, for inner node N1, at which the PVE was higher after SM-DA, compared with the open-loop. When compared to the unperturbed model run (Table ), the assimilation of satellite soil moisture improved the performance of the model in terms of PVE at all the nodes and for both the lumped and semi-distributed schemes.

The skill of the ensembles after SM-DA was improved at the catchment outlet by 12 and 13 % (expressed by a reduction in CRPS) for the lumped and semi-distributed scheme respectively, and by a 17 % at N1. The skill of the updated ensemble was also consistently higher than the unperturbed model run (Table 2).

To summarise the efficacy of the SM-DA, we take into account the characteristics of the ensemble predictions (open-loop and updated) in terms of the their mean, skill and reliability. Overall, SM-DA was effective at improving streamflow ensemble predictions in the gauged and the ungauged catchments. By accounting for rainfall spatial distribution and routing process within the large study catchment, we improved the model performance at the outlet compared with a lumped homogeneous scheme. This led to greater improvements from the SM-DA for the semi-distributed model. The latter was achieved even though the relationship between θ and the streamflow prediction was weaker in the semi-distributed scheme (Fig. ). The proposed SM-DA scheme therefore has the merits of improving streamflow ensemble predictions by correcting the SM state of the model, even when rainfall appears to be the main driver of the runoff mechanism (see Sect. ).