Hybridizing Bayesian and variational data assimilation  for high-resolution hydrologic forecasting

Hernández, Felipe; Liang, Xu

doi:https://doi.org/10.5194/hess-22-5759-2018

Articles | Volume 22, issue 11

https://doi.org/10.5194/hess-22-5759-2018

© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-22-5759-2018

© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 22, issue 11

Research article

|

09 Nov 2018

Research article |

| 09 Nov 2018

Hybridizing Bayesian and variational data assimilation for high-resolution hydrologic forecasting

Felipe Hernández and Xu Liang

Download

Final revised paper (published on 09 Nov 2018)
Supplement to the final revised paper
Preprint (discussion started on 11 Aug 2017)
Supplement to the preprint

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review of "Hybridizing Bayesian and variational data assimilation for robust high-resolution hydrologic forecasting"', Anonymous Referee #1, 13 Oct 2017
- AC3: 'Authors' response to referee #1', Felipe Hernández, 19 Jan 2018
RC2: 'Review of “Hybridizing Bayesian and variational data assimilation for robust high-resolution hydrologic forecasting” by Felipe Hernández and Xu Liang', Anonymous Referee #2, 21 Dec 2017
- AC4: 'Authors' response to referee #2', Felipe Hernández, 19 Jan 2018
EC1: 'Editor's comment', Dimitri Solomatine, 22 Jan 2018

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Publish subject to revisions (further review by editor and referees) (22 Jan 2018) by Dimitri Solomatine

AR by Felipe Hernández on behalf of the Authors (12 Mar 2018) Author's response Manuscript

ED: Referee Nomination & Report Request started (19 Mar 2018) by Dimitri Solomatine

RR by Maurizio Mazzoleni (16 Apr 2018)

RR by Anonymous Referee #2 (23 Apr 2018)

Suggestions for revision or reasons for rejection

I appreciate the authors’ response and the revised manuscript which extended the evaluation period for the low-resolution domain and included one probabilistic measure. I highly evaluate this unified approach to combine two different data assimilation approaches: sequential and variational methods. However, despite the effort, it is still doubtful that the capability and robustness of this new data assimilation approach, OPTIMISTS, are properly demonstrated through numerical experiments. Especially, probabilistic features of this new method are not evaluated, and instability is partly observed in the low flow forecasts.

As shown in the extended analysis illustrated in Fig. 4, the forecasts by OPTIMISTS show unstable behaviors compared to the particle filter (PF), although such instability is not captured through statistical measures used in this study fortunately. I don’t think this instability is evidence to give the new method an edge over traditional approaches (Page 16). Neither common Bayesian nor variational approaches lead to such numerical instability while the authors claimed the simulated result as the combined features of those two approaches.

In the meanwhile, an additional question arises about whether or not model ensemble are properly setup before we discuss data assimilation. Ensemble spreads of PF and OPTIMISTS are extremely narrow in both 6-day and 24-day forecasts, which implies the model uncertainty is not properly represented by ensemble. Even if sampling and regularization steps were devised to avoid sample degeneracy in this study, it is highly likely that the space of state variables could remain narrow and sampling from such narrow space may not lead to optimal solutions whatever kernels or pareto methods are used. I suspect that inappropriate ensemble setup may be to blame for low performance in overall DA simulations.

Although the authors claimed OPTIMISTS efficiently produced ‘probabilistic’ forecasts in the abstract, no evidence is given to prove this claim. CRPS, the only probabilistic measure used in the revised manuscript, just shows the magnitude of bias. Whether or not posterior distributions produced by the OPTIMISTS are appropriate from probabilistic perspectives still remains unproven. It should be also mentioned that non-normality (non-Gaussian feature) is not evaluated because measures used rely on the first two moments (mean and variance). Rather, given 50% ensemble traces in Fig. 4, if the rank histogram (Talagrand diagram) is estimated, U-shaped diagram is expected because the ensemble spread is too small (For perfect probabilistic forecasts, the rank histogram is flat).

Reproducibility of the algorithm is key to hydrologic research. Although the authors claimed details are explained in subsections, I don’t think any hydrologic experts can reproduce the data assimilation procedure by following this manuscript. It remains still vague about how to implement this method for high-dimensional hydrologic modeling. If all are explained as the authors claimed, it may be reasonable to think the current implementation procedure is lacking important part to represent model uncertainty and generate spatial diversity of ensemble.

Hide

ED: Publish subject to revisions (further review by editor and referees) (12 May 2018) by Dimitri Solomatine

AR by Felipe Hernández on behalf of the Authors (23 Jun 2018) Author's response Manuscript

ED: Referee Nomination & Report Request started (28 Jun 2018) by Dimitri Solomatine

RR by Anonymous Referee #2 (03 Aug 2018)

Suggestions for revision or reasons for rejection

In the revised manuscript, all the issues the reviewer raised were addressed: the numerical experiments were improved; the analysis and evaluation were properly adjusted and supported by the evidence. The last question I had was whether we could conclude this new data assimilation method ‘robust’ when the performance of the new method with more complicated procedures is comparable or lower over the conventional simpler method. In fact, I am not positive on this question. However, I am at least confident that the new approach in this study demonstrates the feasibility of hybrid data assimilation on high-resolution modeling and interesting findings from various numerical experiments should be shared with research community. In addition, the developed model code, which is open to public, will be helpful for a wide range of hydrologic community to apply and further improve the algorithm. I am grateful for the efforts of the authors and the handling editor.
I also highly recommend that parts of Author’s Response, which contains important diagnoses and analysis on the developed method, would be included in the final publication if this final response is not open to general readers. The following parts in Author’s Response may be considered to be included in the final publication.

- We found that there was a tradeoff between allowing for some level of “instability” and the scores obtained for the deterministic error measures. In effect, allowing for more “exploration” (and less “exploitation”) leads sometimes to better results thanks to ensemble diversity but sometimes to too much diversity (wider spread).

- OPTIMISTS has the flexibility of being configured to produce forecasts with more variance, but such flexibility should be used with caution as some configurations may result in “instabilities”.

Lastly, different from the description in Page 15 Line 34 and Page 16 Line 1, I don’t think Fig. 4 evidences the non-Gaussian nature. The temporal evolution in Fig. 4, which illustrates the varying ensemble spreads, can also occur under Gaussian assumptions. If you observed the non-Gaussian feature, which is not explicitly shown in the figure, please add one or two sentences in the text.

Hide

ED: Publish subject to minor revisions (review by editor) (23 Aug 2018) by Dimitri Solomatine

AR by Felipe Hernández on behalf of the Authors (29 Aug 2018) Author's response Manuscript

ED: Publish subject to minor revisions (review by editor) (03 Sep 2018) by Dimitri Solomatine

AR by Felipe Hernández on behalf of the Authors (05 Sep 2018) Author's response Manuscript

ED: Publish as is (06 Sep 2018) by Dimitri Solomatine

AR by Felipe Hernández on behalf of the Authors (16 Sep 2018) Author's response Manuscript

Short summary

Predicting floods requires first knowing the amount of water in the valleys, which is complicated because we cannot know for sure how much water there is in the soil. We created a unique system that combines the best methods to estimate these conditions accurately based on the observed water flow in the rivers and on detailed simulations of the valleys. Comparisons with popular methods show that our system can produce realistic predictions efficiently, even for very detailed river networks.