25 Oct 2021
25 Oct 2021
Status: a revised version of this preprint is currently under review for the journal HESS.

Technical Note: Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks

Grey S. Nearing1,2, Daniel Klotz3, Alden Keefe Sampson4, Frederik Kratzert5, Martin Gauch3, Jonathan M. Frame6,7, Guy Shalev8, and Sella Nevo8 Grey S. Nearing et al.
  • 1Google Research, Mountain View, CA, United States
  • 2University of California Davis, Department of Land, Air & Water Resources, Davis, CA, United States
  • 3LIT AI Lab & Institute for Machine Learning, Johannes Kepler University, Linz, Austria
  • 4Upstream Tech, Alameda, CA, USA
  • 5Google Research, Vienna, Austria
  • 6National Water Center, National Oceanic and Atmospheric Administration, Tuscaloosa, AL, United States
  • 7Department of Geological Sciences, University of Alabama, Tuscaloosa, AL, USA
  • 8Google Research, Tel Aviv, Israel

Abstract. Ingesting near-real-time observation data is a critical component of many operational hydrological forecasting systems. In this paper we compare two strategies for ingesting near-real-time streamflow observations into Long Short-Term Memory (LSTM) rainfall-runoff models: autoregression (a forward method) and variational data assimilation. Autoregression is both more accurate and more computationally efficient than data assimilation. Autoregression is sensitive to missing data, however an appropriate (and simple) training strategy mitigates this problem.

Grey S. Nearing et al.

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on hess-2021-515', Ralf Loritz, 25 Nov 2021
  • RC2: 'review of hess-2021-515', Anonymous Referee #2, 02 Dec 2021

Grey S. Nearing et al.

Grey S. Nearing et al.


Total article views: 1,123 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
776 323 24 1,123 12 13
  • HTML: 776
  • PDF: 323
  • XML: 24
  • Total: 1,123
  • BibTeX: 12
  • EndNote: 13
Views and downloads (calculated since 25 Oct 2021)
Cumulative views and downloads (calculated since 25 Oct 2021)

Viewed (geographical distribution)

Total article views: 1,048 (including HTML, PDF, and XML) Thereof 1,048 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 28 Jun 2022
Short summary
When designing flood forecasting models, it is necessary to use all available data to achieve the most accurate predictions possible. This manuscript explores two basic ways of ingesting near-real-time streamflow data into machine learning streamflow models. The point we want to make is that when working in the context of machine learning (instead of traditional hydrology models that are based on bio-geophysics), it is not necessary to use complex statistical methods for injecting sparse data.