|I was asked to look again at a revision of this paper. To be honest, at this point, I am not sure whether I commented on an earlier draft of this paper. I think I did. In short, I looked at the revision and have substantial concerns about the presented work. I did confirm with the Editor that I am reviewing the "right" version of the manuscript as I was surprised seeing relatively few changes to earlier comments on the methodology.|
1. The authors use the word "filtering" - this suggests a state estimation exercise. In practice, no effort is made to discuss/describe the filtering methodology. Likelihood, incremental likelihood, and normalized likelihood. Which state variables are being included in the analysis The responses of the authors' to previous comments on this matter were not helpful as they did not lead to changes in the paper. What is the model error that is used - the stochastic perturbation of the state/output forecast? What is the measurement error of the discharge data that is being used - all this determines the likelihood, incremental likelihood and normalized likelihood of the particles - and so their time evolution. This is all crucial information to understand the implementation and evaluate the results. As I guess the authors pick at each time from a predefined ensemble those parameter vectors that perform "best" for a short past time period - and use those with state estimation to generate a forecast. I am not sure about my interpretation.
2. If filtering is not used re: state estimation then the authors should make this clear. In their algorithmic recipe nothing is mentioned about state estimation - nor details given. So, if filtering refers to picking out the "best" parameters at each time step then this should be made clear. If this is true, then the authors are missing other important work on this topic such as the DYNIA approach - and the PIMLI methodology that have elements in common with what the authors are doing in present paper. What is also relevant is the BARE approach of Thiemann et al. (2000) - Bayesian recursive estimation - also many elements in common with present approach if authors did indeed not do state estimation.
3. The authors implement an ABC methodology. They use the model as likelihood - but no likelihood is presented - OK - if the model output is the likelihood then this does not suffice - the likelihood (simulated discharge) requires perturbation to make the model stochastic rather than deterministic. ABC method cannot work with a deterministic likelihood - otherwise the posterior would converge to a point in the limit of "epsilon" going to zero. So, what is the stochastic perturbation that the authors are using to compute/define the likelihood? One cannot simply use the present model as likelihood. Instead, assumptions are required about the expected probabilistic properties of the model error - and one needs to sample from this distribution to corrupt the deterministic forecasts - this will then converge to the exact target - if the assumptions about the residual errors are honored by the data. See Vrugt and Beven (2018) for more detailed comments on this matter - and our earlier papers published in 2013, 2014 and 2015 in WRR, HESS, etc..
4. The authors use Latin Hyper Cube Sampling to sample 10,000 parameter vectors - and at each time they simple use for their forecasts the best "M" parameter vectors from the past few days. Is this what the authors are doing? This involves no filtering as per state estimation. If state estimation is used on top then I am worried about the initial states of the members that are not used. Lets say that at time 1 one uses members 1-10 to generate a forecast for time 2 - those members were found to produce the best forecasts for time 0. Then state estimation is used for those members 1-10. Then I would expect the initial states at time 2 of these members to be better than those remaining members (11-10000) of the ensemble s their states have not been estimated - so then at time 2 I would expect the first 10 members to do better for time 3 - as their states were estimated - and not those of the other members. So, again, I have simple but important questions about the methodology.
I cannot evaluate the results without understanding in detail the methodology.
I hope these comments are useful to further enhance the manuscript. It may very well be possible that I completely misread/misunderstand the authors intentions. Even then, it may benefit the authors from addressing some of these comments/concerns as other readers may experience similar issues with the presented material.
Irvine, July 2, 2018