the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
25 Oct 2021
25 Oct 2021
Technical Note: Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks
- 1Google Research, Mountain View, CA, United States
- 2University of California Davis, Department of Land, Air & Water Resources, Davis, CA, United States
- 3LIT AI Lab & Institute for Machine Learning, Johannes Kepler University, Linz, Austria
- 4Upstream Tech, Alameda, CA, USA
- 5Google Research, Vienna, Austria
- 6National Water Center, National Oceanic and Atmospheric Administration, Tuscaloosa, AL, United States
- 7Department of Geological Sciences, University of Alabama, Tuscaloosa, AL, USA
- 8Google Research, Tel Aviv, Israel
- 1Google Research, Mountain View, CA, United States
- 2University of California Davis, Department of Land, Air & Water Resources, Davis, CA, United States
- 3LIT AI Lab & Institute for Machine Learning, Johannes Kepler University, Linz, Austria
- 4Upstream Tech, Alameda, CA, USA
- 5Google Research, Vienna, Austria
- 6National Water Center, National Oceanic and Atmospheric Administration, Tuscaloosa, AL, United States
- 7Department of Geological Sciences, University of Alabama, Tuscaloosa, AL, USA
- 8Google Research, Tel Aviv, Israel
Abstract. Ingesting near-real-time observation data is a critical component of many operational hydrological forecasting systems. In this paper we compare two strategies for ingesting near-real-time streamflow observations into Long Short-Term Memory (LSTM) rainfall-runoff models: autoregression (a forward method) and variational data assimilation. Autoregression is both more accurate and more computationally efficient than data assimilation. Autoregression is sensitive to missing data, however an appropriate (and simple) training strategy mitigates this problem.
Grey S. Nearing et al.
Status: open (until 20 Dec 2021)
-
RC1: 'Comment on hess-2021-515', Ralf Loritz, 25 Nov 2021
reply
Review of “Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks” by Nearing et al. (in discussion)
Summary and Recommendation
Nearing et al. test how near-real time streamflow observations can effectively be used in Long Short-Term Memory (LSTM) rainfall runoff models. They compare an autoregression (AR) approach with a data assimilation (DA) approach and test, additionally, how sensitive AR is to random gaps in the data. The manuscript (MS) is easy to follow, well-structured and suits the scope of HESS. I particular liked the comprehensive appendix in combination with a short MS. I think that this MS can be published after some minor revisions and provide only some smaller comments and questions below.
Sincerely
Ralf Loritz
Questions and comments:
- Reading the MS I would have like to see a couple of detailed results from three or four catchments where the AR or DA worked particular good or bad and what the (hydrological and ML) reason for this might be (to underpin the discussion of Appendix F and G). For instance, what could be the reason that DA and AR reduces the predictive performance of a few of your models (Fig. F2)? You state that: "We are unsure of the reason for this, but it warrants further exploration." (Line 352) maybe zooming into one of the catchments could help to give a better explaination.
- I find it a bit unrealistic how you added the missing data. I would assume that a broken gauging station is not working for a couple of days or maybe weeks in a row and wonder how this would alter your results (e.g. all streamflow data available for training but then two weeks or more only simulated data during testing with a closer focus on particular that period and not the entire testing period).
- Showing how the variance or the Shannon entropy changes of your simulations in addition to the median would be interesting (Fig.1 and Fig.3). If it remains constant, I would mention that the spread of the predictions is not affected by the data availability.
Personal comment: Three of the seven Co-Authors have presumingly not contributed to this “technical note” as they are not mentioned in the author contribution section.
Grey S. Nearing et al.
Grey S. Nearing et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 381 | 126 | 5 | 512 | 1 | 2 |
- HTML: 381
- PDF: 126
- XML: 5
- Total: 512
- BibTeX: 1
- EndNote: 2
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1