Articles | Volume 29, issue 23
https://doi.org/10.5194/hess-29-6811-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/hess-29-6811-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Research article
| Highlight paper
| 
01 Dec 2025
Research article | Highlight paper |
| 01 Dec 2025
| 01 Dec 2025
From RNNs to Transformers: benchmarking deep learning architectures for hydrologic prediction
Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA
Chaopeng Shen
Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA
Fearghal O'Donncha
IBM Research, Dublin, Ireland
Yalan Song
Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA
Wei Zhi
Hohai University, Nanjing, China
Hylke E. Beck
King Abdullah University of Science and Technology, Thuwal, Saudi Arabia
Tadd Bindas
Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA
Nicholas Kraabel
Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA
Kathryn Lawson
Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA
Related authors
Dapeng Feng, Hylke Beck, Jens de Bruijn, Reetik Kumar Sahu, Yusuke Satoh, Yoshihide Wada, Jiangtao Liu, Ming Pan, Kathryn Lawson, and Chaopeng Shen
Geosci. Model Dev., 17, 7181–7198, https://doi.org/10.5194/gmd-17-7181-2024, https://doi.org/10.5194/gmd-17-7181-2024, 2024
Short summary
Short summary
Accurate hydrologic modeling is vital to characterizing water cycle responses to climate change. For the first time at this scale, we use differentiable physics-informed machine learning hydrologic models to simulate rainfall–runoff processes for 3753 basins around the world and compare them with purely data-driven and traditional modeling approaches. This sets a benchmark for hydrologic estimates around the world and builds foundations for improving global hydrologic simulations.
Doaa Aboelyazeed, Chonggang Xu, Forrest M. Hoffman, Jiangtao Liu, Alex W. Jones, Chris Rackauckas, Kathryn Lawson, and Chaopeng Shen
Biogeosciences, 20, 2671–2692, https://doi.org/10.5194/bg-20-2671-2023, https://doi.org/10.5194/bg-20-2671-2023, 2023
Short summary
Short summary
Photosynthesis is critical for life and has been affected by the changing climate. Many parameters come into play while modeling, but traditional calibration approaches face many issues. Our framework trains coupled neural networks to provide parameters to a photosynthesis model. Using big data, we independently found parameter values that were correlated with those in the literature while giving higher correlation and reduced biases in photosynthesis rates.
Jiangtao Liu, David Hughes, Farshid Rahmani, Kathryn Lawson, and Chaopeng Shen
Geosci. Model Dev., 16, 1553–1567, https://doi.org/10.5194/gmd-16-1553-2023, https://doi.org/10.5194/gmd-16-1553-2023, 2023
Short summary
Short summary
Under-monitored regions like Africa need high-quality soil moisture predictions to help with food production, but it is not clear if soil moisture processes are similar enough around the world for data-driven models to maintain accuracy. We present a deep-learning-based soil moisture model that learns from both in situ data and satellite data and performs better than satellite products at the global scale. These results help us apply our model globally while better understanding its limitations.
Peijun Li, Yalan Song, Ming Pan, Kathryn Lawson, and Chaopeng Shen
Hydrol. Earth Syst. Sci., 29, 6829–6861, https://doi.org/10.5194/hess-29-6829-2025, https://doi.org/10.5194/hess-29-6829-2025, 2025
Short summary
Short summary
This study explores how combining different model types improves streamflow predictions, especially in data-sparse scenarios. By integrating two highly accurate models with distinct mechanisms and leveraging multiple meteorological datasets, we highlight their unique strengths and set new accuracy benchmarks across spatiotemporal conditions. Our findings enhance the understanding of how diverse models and multi-source data can be effectively used to improve hydrological predictions.
Yuan Yang, Ming Pan, Dapeng Feng, Mu Xiao, Taylor Dixon, Robert Hartman, Chaopeng Shen, Yalan Song, Agniv Sengupta, Luca Delle Monache, and F. Martin Ralph
Hydrol. Earth Syst. Sci., 29, 5453–5476, https://doi.org/10.5194/hess-29-5453-2025, https://doi.org/10.5194/hess-29-5453-2025, 2025
Short summary
Short summary
We explore a machine learning-based data integration method that integrates streamflow (Q) and snow water equivalent (SWE) to improve streamflow estimates at various lag times (1–10 d, 1–6 months) and timescales (daily and monthly) over Western US basins. Benefits rank as: integrating Q at the daily scale > Q at the monthly scale > SWE at the monthly scale > SWE at the daily scale. Results highlight the method’s potential for short- and long-term streamflow forecasting in the Western US.
Wouter J. M. Knoben, Ashwin Raman, Gaby J. Gründemann, Mukesh Kumar, Alain Pietroniro, Chaopeng Shen, Yalan Song, Cyril Thébault, Katie van Werkhoven, Andrew W. Wood, and Martyn P. Clark
Hydrol. Earth Syst. Sci., 29, 2361–2375, https://doi.org/10.5194/hess-29-2361-2025, https://doi.org/10.5194/hess-29-2361-2025, 2025
Short summary
Short summary
Hydrologic models are needed to provide simulations of water availability, floods, and droughts. The accuracy of these simulations is often quantified with so-called performance scores. A common thought is that different models are more or less applicable to different landscapes, depending on how the model works. We show that performance scores are not helpful in distinguishing between different models and thus cannot easily be used to select an appropriate model for a specific place.
Mohammad Sina Jahangir, John Quilty, Chaopeng Shen, Andrea Scott, Scott Steinschneider, and Jan Adamowski
EGUsphere, https://doi.org/10.5194/egusphere-2025-846, https://doi.org/10.5194/egusphere-2025-846, 2025
Short summary
Short summary
This study presents a novel hybrid approach to streamflow prediction, significantly improving the efficiency and accuracy of fine-tuning deep learning models for hydrological prediction. Tested across numerous catchments in the U.S. and Europe, this method accelerates the fine-tuning process and improves prediction accuracy in locations beyond the training data. This innovative approach sets the stage for future hydrological models leveraging transfer learning.
Ather Abbas, Yuan Yang, Ming Pan, Yves Tramblay, Chaopeng Shen, Haoyu Ji, Solomon H. Gebrechorkos, Florian Pappenberger, Jong Cheol Pyo, Dapeng Feng, George Huffman, Phu Nguyen, Christian Massari, Luca Brocca, Tan Jackson, and Hylke E. Beck
EGUsphere, https://doi.org/10.5194/egusphere-2024-4194, https://doi.org/10.5194/egusphere-2024-4194, 2025
Short summary
Short summary
Our study evaluated 23 precipitation datasets using a hydrological model at global scale to assess their suitability and accuracy. We found that MSWEP V2.8 excels due to its ability to integrate data from multiple sources, while others, such as IMERG and JRA-3Q, demonstrated strong regional performances. This research assists in selecting the appropriate dataset for applications in water resource management, hazard assessment, agriculture, and environmental monitoring.
Dapeng Feng, Hylke Beck, Jens de Bruijn, Reetik Kumar Sahu, Yusuke Satoh, Yoshihide Wada, Jiangtao Liu, Ming Pan, Kathryn Lawson, and Chaopeng Shen
Geosci. Model Dev., 17, 7181–7198, https://doi.org/10.5194/gmd-17-7181-2024, https://doi.org/10.5194/gmd-17-7181-2024, 2024
Short summary
Short summary
Accurate hydrologic modeling is vital to characterizing water cycle responses to climate change. For the first time at this scale, we use differentiable physics-informed machine learning hydrologic models to simulate rainfall–runoff processes for 3753 basins around the world and compare them with purely data-driven and traditional modeling approaches. This sets a benchmark for hydrologic estimates around the world and builds foundations for improving global hydrologic simulations.
Yalan Song, Wouter J. M. Knoben, Martyn P. Clark, Dapeng Feng, Kathryn Lawson, Kamlesh Sawadekar, and Chaopeng Shen
Hydrol. Earth Syst. Sci., 28, 3051–3077, https://doi.org/10.5194/hess-28-3051-2024, https://doi.org/10.5194/hess-28-3051-2024, 2024
Short summary
Short summary
Differentiable models (DMs) integrate neural networks and physical equations for accuracy, interpretability, and knowledge discovery. We developed an adjoint-based DM for ordinary differential equations (ODEs) for hydrological modeling, reducing distorted fluxes and physical parameters from errors in models that use explicit and operation-splitting schemes. With a better numerical scheme and improved structure, the adjoint-based DM matches or surpasses long short-term memory (LSTM) performance.
Peter Reichert, Kai Ma, Marvin Höge, Fabrizio Fenicia, Marco Baity-Jesi, Dapeng Feng, and Chaopeng Shen
Hydrol. Earth Syst. Sci., 28, 2505–2529, https://doi.org/10.5194/hess-28-2505-2024, https://doi.org/10.5194/hess-28-2505-2024, 2024
Short summary
Short summary
We compared the predicted change in catchment outlet discharge to precipitation and temperature change for conceptual and machine learning hydrological models. We found that machine learning models, despite providing excellent fit and prediction capabilities, can be unreliable regarding the prediction of the effect of temperature change for low-elevation catchments. This indicates the need for caution when applying them for the prediction of the effect of climate change.
Doaa Aboelyazeed, Chonggang Xu, Forrest M. Hoffman, Jiangtao Liu, Alex W. Jones, Chris Rackauckas, Kathryn Lawson, and Chaopeng Shen
Biogeosciences, 20, 2671–2692, https://doi.org/10.5194/bg-20-2671-2023, https://doi.org/10.5194/bg-20-2671-2023, 2023
Short summary
Short summary
Photosynthesis is critical for life and has been affected by the changing climate. Many parameters come into play while modeling, but traditional calibration approaches face many issues. Our framework trains coupled neural networks to provide parameters to a photosynthesis model. Using big data, we independently found parameter values that were correlated with those in the literature while giving higher correlation and reduced biases in photosynthesis rates.
Dapeng Feng, Hylke Beck, Kathryn Lawson, and Chaopeng Shen
Hydrol. Earth Syst. Sci., 27, 2357–2373, https://doi.org/10.5194/hess-27-2357-2023, https://doi.org/10.5194/hess-27-2357-2023, 2023
Short summary
Short summary
Powerful hybrid models (called δ or delta models) embrace the fundamental learning capability of AI and can also explain the physical processes. Here we test their performance when applied to regions not in the training data. δ models rivaled the accuracy of state-of-the-art AI models under the data-dense scenario and even surpassed them for the data-sparse one. They generalize well due to the physical structure included. δ models could be ideal candidates for global hydrologic assessment.
Louise J. Slater, Louise Arnal, Marie-Amélie Boucher, Annie Y.-Y. Chang, Simon Moulds, Conor Murphy, Grey Nearing, Guy Shalev, Chaopeng Shen, Linda Speight, Gabriele Villarini, Robert L. Wilby, Andrew Wood, and Massimiliano Zappa
Hydrol. Earth Syst. Sci., 27, 1865–1889, https://doi.org/10.5194/hess-27-1865-2023, https://doi.org/10.5194/hess-27-1865-2023, 2023
Short summary
Short summary
Hybrid forecasting systems combine data-driven methods with physics-based weather and climate models to improve the accuracy of predictions for meteorological and hydroclimatic events such as rainfall, temperature, streamflow, floods, droughts, tropical cyclones, or atmospheric rivers. We review recent developments in hybrid forecasting and outline key challenges and opportunities in the field.
Jiangtao Liu, David Hughes, Farshid Rahmani, Kathryn Lawson, and Chaopeng Shen
Geosci. Model Dev., 16, 1553–1567, https://doi.org/10.5194/gmd-16-1553-2023, https://doi.org/10.5194/gmd-16-1553-2023, 2023
Short summary
Short summary
Under-monitored regions like Africa need high-quality soil moisture predictions to help with food production, but it is not clear if soil moisture processes are similar enough around the world for data-driven models to maintain accuracy. We present a deep-learning-based soil moisture model that learns from both in situ data and satellite data and performs better than satellite products at the global scale. These results help us apply our model globally while better understanding its limitations.
Cited articles
Aboelyazeed, D., Xu, C., Hoffman, F. M., Liu, J., Jones, A. W., Rackauckas, C., Lawson, K., and Shen, C.: A differentiable, physics-informed ecosystem modeling and learning framework for large-scale inverse problems: demonstration with photosynthesis simulations, Biogeosciences, 20, 2671–2692, https://doi.org/10.5194/bg-20-2671-2023, 2023.
Aboelyazeed, D., Xu, C., Gu, L., Luo, X., Liu, J., Lawson, K., and Shen, C.: Inferring plant acclimation and improving model generalizability with differentiable physics-informed machine learning of photosynthesis, J. Geophys. Res.- Biogeosci., 130, e2024JG008552, https://doi.org/10.1029/2024JG008552, 2025.
Addor, N., Newman, A. J., Mizukami, N., and Clark, M. P.: Catchment Attributes and MEteorology for Large-Sample studies (CAMELS) version 2.0, UCAR/NCAR [dataset], https://doi.org/10.5065/D6G73C3Q, 2017.
Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., Zschiegner, J., Maddix, D. C., Wang, H., Mahoney, M. W., Torkkola, K., Wilson, A. G., Bohlke-Schneider, M., and Wang, Y.: Chronos: Learning the language of time series, arXiv [preprint], https://doi.org/10.48550/arXiv.2403.07815, 4 November 2024.
Bai, P., Liu, X., Yang, T., Liang, K., and Liu, C.: Evaluation of streamflow simulation results of land surface models in GLDAS on the Tibetan plateau, Journal of Geophysical Research: Atmospheres, 121, 12180-12197, https://doi.org/10.1002/2016JD025501, 2016.
Basso, S., Merz, R., Tarasova, L., and Miniussi, A.: Extreme flooding controlled by stream network organization and flow regime, Nature Geoscience, 16, 339–343, https://doi.org/10.1038/s41561-023-01155-w, 2023.
Bayissa, Y., Melesse, A., Bhat, M., Tadesse, T., and Shiferaw, A.: Evaluation of regional climate models (RCMs) using precipitation and temperature-based climatic indices: A case study of Florida, USA, Water, 13, 2411, https://doi.org/10.3390/w13172411, 2021.
Beck, H. E., Pan, M., Lin, P., Seibert, J., van Dijk, A. I. J. M., and Wood, E. F.: Global fully distributed parameter regionalization based on observed streamflow from 4,229 headwater catchments, Journal of Geophysical Research-Atmospheres, 125, e2019JD031485, https://doi.org/10.1029/2019JD031485, 2020.
Bilokon, P. and Qiu, Y.: Transformers versus LSTMs for electronic trading, arXiv [preprint], https://doi.org/10.48550/arXiv.2309.11400, 20 September 2023.
Brunner, L. and Voigt, A.: Pitfalls in diagnosing temperature extremes, Nature Communications, 15, 2087, https://doi.org/10.1038/s41467-024-46349-x, 2024.
Castangia, M., Grajales, L. M. M., Aliberti, A., Rossi, C., Macii, A., Macii, E., and Patti, E.: Transformer neural networks for interpretable flood forecasting, Environmental Modelling & Software, 160, 105581, https://doi.org/10.1016/j.envsoft.2022.105581, 2023.
Delforge, D., de Viron, O., Vanclooster, M., Van Camp, M., and Watlet, A.: Detecting hydrological connectivity using causal inference from time series: synthetic and real karstic case studies, Hydrol. Earth Syst. Sci., 26, 2181–2199, https://doi.org/10.5194/hess-26-2181-2022, 2022.
Dorigo, W. A., Wagner, W., Hohensinn, R., Hahn, S., Paulik, C., Xaver, A., Gruber, A., Drusch, M., Mecklenburg, S., van Oevelen, P., Robock, A., and Jackson, T.: The International Soil Moisture Network: a data hosting facility for global in situ soil moisture measurements, Hydrol. Earth Syst. Sci., 15, 1675–1698, https://doi.org/10.5194/hess-15-1675-2011, 2011.
Dorigo, W. A., Xaver, A., Vreugdenhil, M., Gruber, A., Hegyiová, A., Sanchis-Dufau, A. D., Zamojski, D., Cordes, C., Wagner, W., and Drusch, M.: Global automated quality control of in situ soil moisture data from the international soil moisture network, Vadose Zone Journal, 12, vzj2012.0097, https://doi.org/10.2136/vzj2012.0097, 2013.
Ekambaram, V., Jati, A., Dayama, P., Mukherjee, S., Nguyen, N. H., Gifford, W. M., Reddy, C., and Kalagnanam, J.: Tiny Time Mixers (TTMs): Fast pre-trained models for enhanced zero/few-shot forecasting of multivariate time series, arXiv [preprint], https://doi.org/10.48550/arXiv.2401.03955, 7 November 2024.
Faghih, M. and Brissette, F.: Temporal and spatial amplification of extreme rainfall and extreme floods in a warmer climate, Journal of Hydrometeorology, 24, 1331–1347, https://doi.org/10.1175/JHM-D-22-0224.1, 2023.
Fang, K. and Shen, C.: Near-real-time forecast of satellite-based soil moisture using long short-term memory with an adaptive data integration kernel, Journal of Hydrometeorology, 21, 399–413, https://doi.org/10.1175/jhm-d-19-0169.1, 2020.
Fang, K., Shen, C., Kifer, D., and Yang, X.: Prolongation of SMAP to spatiotemporally seamless coverage of continental U.S. using a deep learning neural network, Geophys. Res. Lett., 44, 11030-11039, https://doi.org/10.1002/2017gl075619, 2017.
Feng, D., Fang, K., and Shen, C.: Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales, Water Resources Research, 56, e2019WR026793, https://doi.org/10.1029/2019WR026793, 2020.
Feng, D., Liu, J., Lawson, K., and Shen, C.: Differentiable, learnable, regionalized process-based models with multiphysical outputs can approach state-of-the-art hydrologic prediction accuracy, Water Resources Research, 58, e2022WR032404, https://doi.org/10.1029/2022WR032404, 2022.
Feng, D., Beck, H., Lawson, K., and Shen, C.: The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment, Hydrol. Earth Syst. Sci., 27, 2357–2373, https://doi.org/10.5194/hess-27-2357-2023, 2023.
Feng, D., Beck, H., de Bruijn, J., Sahu, R. K., Satoh, Y., Wada, Y., Liu, J., Pan, M., Lawson, K., and Shen, C.: Deep dive into hydrologic simulations at global scale: harnessing the power of deep learning and physics-informed differentiable models (δHBV-globe1.0-hydroDL), Geosci. Model Dev., 17, 7181–7198, https://doi.org/10.5194/gmd-17-7181-2024, 2024.
Gao, Z., Cui, X., Zhuo, T., Cheng, Z., Liu, A.-A., Wang, M., and Chen, S.: A Multitemporal Scale and Spatial–Temporal Transformer Network for Temporal Action Localization, IEEE Transactions on Human-Machine Systems, 53, 569–580, https://doi.org/10.1109/THMS.2023.3266037, 2023.
Garza, A., Challu, C., and Mergenthaler-Canseco, M.: TimeGPT-1, arXiv [preprint], https://doi.org/10.48550/arXiv.2310.03589, 27 May 2024.
Geneva, N. and Zabaras, N.: Transformers for modeling physical systems, Neural Networks, 146, 272–289, https://doi.org/10.1016/j.neunet.2021.11.022, 2022.
Ghaneei, P., Foroumandi, E., and Moradkhani, H.: Enhancing streamflow prediction in ungauged basins using a nonlinear knowledge-based framework and deep learning, Water Resources Research, 60, e2024WR037152, https://doi.org/10.1029/2024WR037152, 2024.
Ghobadi, F. and Kang, D.: Improving long-term streamflow prediction in a poorly gauged basin using geo-spatiotemporal mesoscale data and attention-based deep learning: A comparative study, Journal of Hydrology, 615, 128608, https://doi.org/10.1016/j.jhydrol.2022.128608, 2022.
Google: Gemini 1 Pro, Google AI [software], https://ai.google.dev/gemini (last access: 22 May 2024), 2023.
Grattafiori, A., Dubey, A., Jauhri, A. et al.: The Llama 3 Herd of Models, arXiv [preprint], https://doi.org/10.48550/arXiv.2407.21783, 23 November 2024.
Gruver, N., Finzi, M., Qiu, S., and Wilson, A. G.: Large language models are zero-shot time series forecasters, arXiv [preprint], https://doi.org/10.48550/arXiv.2310.07820, 12 August 2024.
Gupta, H. V., Kling, H., Yilmaz, K. K., and Martinez, G. F.: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling, Journal of Hydrology, 377, 80–91, https://doi.org/10.1016/j.jhydrol.2009.08.003, 2009.
Hochreiter, S. and Schmidhuber, J.: Long Short-Term Memory, Neural Computation, 9, 1735–1780, https://doi.org/10.1162/neco.1997.9.8.1735, 1997.
Hogikyan, A. and Resplandy, L.: Hydrological cycle amplification imposes spatial patterns on the climate change response of ocean pH and carbonate chemistry, Biogeosciences, 21, 4621–4636, https://doi.org/10.5194/bg-21-4621-2024, 2024.
Jung, H. C., Getirana, A., Arsenault, K. R., Kumar, S., and Maigary, I.: Improving surface soil moisture estimates in West Africa through GRACE data assimilation, Journal of Hydrology, 575, 192–201, https://doi.org/10.1016/j.jhydrol.2019.05.042, 2019.
Kirchner, J. W.: Characterizing nonlinear, nonstationary, and heterogeneous hydrologic behavior using ensemble rainfall–runoff analysis (ERRA): proof of concept, Hydrol. Earth Syst. Sci., 28, 4427–4454, https://doi.org/10.5194/hess-28-4427-2024, 2024.
Kitaev, N., Kaiser, Ł., and Levskaya, A.: Reformer: The efficient Transformer, arXiv [preprint], https://doi.org/10.48550/arXiv.2001.04451, 18 February 2020.
Koya, S. R. and Roy, T.: Temporal Fusion Transformers for streamflow prediction: Value of combining attention with recurrence, Journal of Hydrology, 637, 131301, https://doi.org/10.1016/j.jhydrol.2024.131301, 2024.
Kratzert, F., Klotz, D., Brenner, C., Schulz, K., and Herrnegger, M.: Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks, Hydrol. Earth Syst. Sci., 22, 6005–6022, https://doi.org/10.5194/hess-22-6005-2018, 2018.
Kratzert, F., Klotz, D., Shalev, G., Klambauer, G., Hochreiter, S., and Nearing, G.: Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets, Hydrol. Earth Syst. Sci., 23, 5089–5110, https://doi.org/10.5194/hess-23-5089-2019, 2019.
Kratzert, F., Klotz, D., Hochreiter, S., and Nearing, G. S.: A note on leveraging synergy in multiple meteorological data sets with deep learning for rainfall–runoff modeling, Hydrol. Earth Syst. Sci., 25, 2685–2703, https://doi.org/10.5194/hess-25-2685-2021, 2021.
Lacoste, A., Luccioni, A., Schmidt, V., and Dandres, T.: Quantifying the carbon emissions of machine learning, arXiv [preprint], https://doi.org/10.48550/arXiv.1910.09700, 4 November 2019.
Liu, J.: Interactive Hydrological Model Evaluation Dashboard, Zenodo [dataset], https://doi.org/10.5281/zenodo.17693315, 2025a.
Liu, J. and Shen, C.: From RNNs to Transformers: benchmarking deep learning architectures for hydrologic prediction, Zenodo [code], https://doi.org/10.5281/zenodo.15852145, 2025.
Liu, J., Xu, Z., Bai, J., Peng, D., and Ren, M.: Assessment and correction of the PERSIANN-CDR product in the Yarlung Zangbo River Basin, China, Remote Sensing, 10, 2031, https://doi.org/10.3390/rs10122031, 2018.
Liu, J., Rahmani, F., Lawson, K., and Shen, C.: A multiscale deep learning model for soil moisture integrating satellite and in situ data, Geophysical Research Letters, 49, e2021GL096847, https://doi.org/10.1029/2021GL096847, 2022a.
Liu, J., Hughes, D., Rahmani, F., Lawson, K., and Shen, C.: Evaluating a global soil moisture dataset from a multitask model (GSM3 v1.0) with potential applications for crop threats, Geosci. Model Dev., 16, 1553–1567, https://doi.org/10.5194/gmd-16-1553-2023, 2023.
Liu, J., Bian, Y., Lawson, K., and Shen, C.: Probing the limit of hydrologic predictability with the Transformer network, Journal of Hydrology, 637, 131389, https://doi.org/10.1016/j.jhydrol.2024.131389, 2024a.
Liu, J., Shen, C., Pei, T., Kifer, D., and Lawson, K.: The value of terrain pattern, high-resolution data and ensemble modeling for landslide susceptibility prediction, Journal of Geophysical Research: Machine Learning and Computation, 2, e2024JH000460, https://doi.org/10.1029/2024JH000460, 2025.
Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A. X., and Dustdar, S.: Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting, in: The Tenth International Conference on Learning Representations (ICLR 2022), virtual conference, 25–29 April 2022, https://doi.org/10.34726/2945, 2022b.
Liu, Y., Wu, H., Wang, J., and Long, M.: Non-stationary transformers: Exploring the stationarity in time series forecasting, Advances in Neural Information Processing Systems, 35, 9881–9893, https://doi.org/10.48550/arXiv.2205.14415, 2022c.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M.: iTransformer: Inverted transformers are effective for time series forecasting, arXiv [preprint], https://doi.org/10.48550/arXiv.2310.06625, 14 March 2024b.
Ma, K., Feng, D., Lawson, K., Tsai, W.-P., Liang, C., Huang, X., Sharma, A., and Shen, C.: Transferring hydrologic data across continents – Leveraging data-rich regions to improve hydrologic prediction in data-sparse regions, Water Resources Research, 57, e2020WR028600, https://doi.org/10.1029/2020wr028600, 2021.
Maurer, E. P., Wood, A. W., Adam, J. C., Lettenmaier, D. P., and Nijssen, B.: A long-term hydrologically based dataset of land surface fluxes and states for the conterminous United States, Journal of Climate, 15, 3237–3251, https://doi.org/10.1175/1520-0442(2002)015<3237:ALTHBD>2.0.CO;2, 2002.
Nash, J. E. and Sutcliffe, J. V.: River flow forecasting through conceptual models part I – A discussion of principles, Journal of Hydrology, 10, 282–290, https://doi.org/10.1016/0022-1694(70)90255-6, 1970.
Newman, A., Clark, M., Bock, A., Viger, R., Blodgett, D., Addor, N., and Mizukami, M.: A large-sample watershed-scale hydrometeorological dataset for the contiguous USA, UCAR/NCAR [dataset], https://doi.org/10.5065/D6MW2F4D, 2014.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J.: A time series is worth 64 words: Long-term forecasting with transformers, arXiv [preprint], https://doi.org/10.48550/arXiv.2211.14730, 5 March 2023.
OpenAI: ChatGPT (Nov 14 version), OpenAI [software], https://chat.openai.com (last access: 22 May 2024), 2023.
Orozco López, E., Kaplan, D., and Linhoss, A.: Interpretable transformer neural network prediction of diverse environmental time series using weather forecasts, Water Resources Research, 60, e2023WR036337, https://doi.org/10.1029/2023WR036337, 2024.
Parisotto, E., Song, F., Rae, J., Pascanu, R., Gulcehre, C., Jayakumar, S., Jaderberg, M., Kaufman, R. L., Clark, A., and Noury, S.: Stabilizing transformers for reinforcement learning, in: International conference on machine learning, 7487–7498, https://doi.org/10.48550/arXiv.1910.06764, 2020.
Pham, H., Dai, Z., Ghiasi, G., Kawaguchi, K., Liu, H., Yu, A. W., Yu, J., Chen, Y.-T., Luong, M.-T., Wu, Y., Tan, M., and Le, Q. V.: Combined scaling for zero-shot transfer learning, Neurocomputing, 555, 126658, https://doi.org/10.1016/j.neucom.2023.126658, 2023.
Pölz, A., Blaschke, A. P., Komma, J., Farnleitner, A. H., and Derx, J.: Transformer versus LSTM: A comparison of deep learning models for karst spring discharge forecasting, Water Resources Research, 60, e2022WR032602, https://doi.org/10.1029/2022WR032602, 2024.
Rasul, K., Ashok, A., Williams, A. R., Ghonia, H., Bhagwatkar, R., Khorasani, A., Bayazi, M. J. D., Adamopoulos, G., Riachi, R., Hassen, N., Biloš, M., Garg, S., Schneider, A., Chapados, N., Drouin, A., Zantedeschi, V., Nevmyvaka, Y., and Rish, I.: Lag-Llama: towards foundation models for probabilistic time series forecasting, arXiv [preprint], https://doi.org/10.48550/arXiv.2310.08278, 8 February 2024.
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., and Prabhat: Deep learning and process understanding for data-driven Earth system science, Nature, 566, 195–204, https://doi.org/10/gfvhxk, 2019.
Shen, C.: Deep learning: A next-generation big-data approach for hydrology, Eos, 99, https://doi.org/10.1029/2018EO095649, 2018.
Song, Y., Tsai, W.-P., Gluck, J., Rhoades, A., Zarzycki, C., McCrary, R., Lawson, K., and Shen, C.: LSTM-based data integration to improve snow water equivalent prediction and diagnose error sources, Journal of Hydrometeorology, 25, 223–237, https://doi.org/10.1175/JHM-D-22-0220.1, 2024.
Song, Y., Bindas, T., Shen, C., Ji, H., Knoben, W. J. M., Lonzarich, L., Clark, M. P., Liu, J., van Werkhoven, K., Lamont, S., Denno, M., Pan, M., Yang, Y., Rapp, J., Kumar, M., Rahmani, F., Thébault, C., Adkins, R., Halgren, J., Patel, T., Patel, A., Sawadekar, K. A., and Lawson, K.: High-resolution national-scale water modeling is enhanced by multiscale differentiable physics-informed machine learning, Water Resour. Res., 61, e2024WR038928, https://doi.org/10.1029/2024WR038928, 2025.
Sterle, G., Perdrial, J., Kincaid, D. W., Underwood, K. L., Rizzo, D. M., Haq, I. U., Li, L., Lee, B. S., Adler, T., Wen, H., Middleton, H. and Harpold, A. A.: CAMELS-Chem: augmenting CAMELS (Catchment Attributes and Meteorology for Large-sample Studies) with atmospheric and stream water chemistry data, Hydrol. Earth Syst. Sci., 28, 611–630, https://doi.org/10.5194/hess-28-611-2024, 2024.
Sun, W., Chang, L.-C., and Chang, F.-J.: Deep dive into predictive excellence: Transformer's impact on groundwater level prediction, Journal of Hydrology, 636, 131250, https://doi.org/10.1016/j.jhydrol.2024.131250, 2024.
Tan, M., Merrill, M. A., Gupta, V., Althoff, T., and Hartvigsen, T.: Are language models actually useful for time series forecasting?, arXiv [preprint], https://doi.org/10.48550/arXiv.2406.16964, 26 October 2024.
Thornton, P. E., Running, S. W., and White, M. A.: Generating surfaces of daily meteorological variables over large regions of complex terrain, Journal of Hydrology, 190, 214–251, https://doi.org/10.1016/S0022-1694(96)03128-9, 1997.
Tsai, W.-P., Feng, D., Pan, M., Beck, H., Lawson, K., Yang, Y., Liu, J., and Shen, C.: From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling, Nat. Commun., 12, 5988, https://doi.org/10.1038/s41467-021-26107-z, 2021.
USDA NRCS NWCC: Snow Telemetry (SNOTEL) and Snow Course Data & Products, USDA Natural Resources Conservation Service National Water and Climate Center [dataset], https://www.nrcs.usda.gov/wps/portal/wcc/home/snowClimateMonitoring/snotel (last access: 17 April 2024), 2021. 2021.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I.: Attention is all you need, Advances in Neural Information Processing Systems, 30, 5998–6008, https://doi.org/10.48550/arXiv.1706.03762, 2017.
Vu, D.-Q., Mai, S. T., and Dang, T. D.: Streamflow prediction in the Mekong River Basin using deep neural networks, IEEE Access, https://doi.org/10.1109/ACCESS.2023.3301153, 2023.
Wang, W., Zheng, V. W., Yu, H., and Miao, C.: A survey of zero-shot learning: Settings, methods, and applications, ACM Trans. Intell. Syst. Technol., 10, 13:1–13:37, https://doi.org/10.1145/3293318, 2019.
Wang, Y. and Zha, Y.: Comparison of transformer, LSTM and coupled algorithms for soil moisture prediction in shallow-groundwater-level areas with interpretability analysis, Agricultural Water Management, 305, 109120, https://doi.org/10.1016/j.agwat.2024.109120, 2024.
Wang, Y., Shi, L., Hu, Y., Hu, X., Song, W., and Wang, L.: A comprehensive study of deep learning for soil moisture prediction, Hydrol. Earth Syst. Sci., 28, 917–943, https://doi.org/10.5194/hess-28-917-2024, 2024a.
Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M., and Wang, J.: Deep time series models: A comprehensive survey and benchmark, https://doi.org/10.48550/arXiv.2407.13278, 18 July, 2024b.
Wang, Z., Bai, Y., Zhou, Y., and Xie, C.: Can CNNs be more robust than transformers?, https://doi.org/10.48550/arXiv.2206.03452, 6 March 2023.
Wessel, J. B., Ferro, C. A. T., and Kwasniok, F.: Lead-time-continuous statistical postprocessing of ensemble weather forecasts, Q. J. Roy. Meteor. Soc., 150, 2147–2167, https://doi.org/10.1002/qj.4701, 2024.
Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S.: ETSformer: Exponential smoothing transformers for time-series forecasting, arXiv [preprint], https://doi.org/10.48550/arXiv.2202.01381, 20 June 2022.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M.: Timesnet: Temporal 2d-variation modeling for general time series analysis, arXiv [preprint], https://doi.org/10.48550/arXiv.2210.02186, 2022.
Xia, Y., Mitchell, K., Ek, M., Sheffield, J., Cosgrove, B., Wood, E., Luo, L., Alonge, C., Wei, H., Meng, J., Livneh, B., Lettenmaier, D., Koren, V., Duan, Q., Mo, K., Fan, Y., and Mocko, D.: Continental-scale water and energy flux analysis and validation for the North American Land Data Assimilation System project phase 2 (NLDAS-2): 1. Intercomparison and application of model products, Journal of Geophysical Research: Atmospheres, 117, https://doi.org/10.1029/2011JD016048, 2012.
Xue, W., Zhou, T., Wen, Q., Gao, J., Ding, B., and Jin, R.: CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting, arXiv [preprint], https://doi.org/10.48550/arXiv.2305.12095, 16 February 2024.
Yilmaz, K. K., Gupta, H. V., and Wagener, T.: A process-based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model, Water Resources Research, 44, https://doi.org/10.1029/2007WR006716, 2008.
Yin, H., Zhu, W., Zhang, X., Xing, Y., Xia, R., Liu, J., and Zhang, Y.: Runoff predictions in new-gauged basins using two transformer-based models, Journal of Hydrology, 622, 129684, https://doi.org/10.1016/j.jhydrol.2023.129684, 2023.
Zeng, A., Chen, M., Zhang, L., and Xu, Q.: Are transformers effective for time series forecasting?, arXiv [preprint], https://doi.org/10.48550/arXiv.2205.13504, 17 August 2022.
Zhang, J., Guan, K., Peng, B., Pan, M., Zhou, W., Jiang, C., Kimm, H., Franz, T. E., Grant, R. F., Yang, Y., Rudnick, D. R., Heeren, D. M., Suyker, A. E., Bauerle, W. L., and Miner, G. L.: Sustainable irrigation based on co-regulation of soil water supply and atmospheric evaporative demand, Nat Commun, 12, 5549, https://doi.org/10.1038/s41467-021-25254-7, 2021.
Zhang, X., Chowdhury, R. R., Gupta, R. K., and Shang, J.: Large language models for time series: A survey, arXiv [preprint], https://doi.org/10.48550/arXiv.2402.01801, 6 May 2024.
Zhang, Y. and Yan, J.: Crossformer: transformer utilizing cross-dimension dependency for multivariate time series forecasting, in: The Eleventh International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda, 1–5 May 2023, OpenReview, https://openreview.net/forum?id=vSVLM2j9eie (last access: 2 May 2024), 2023.
Zhang, Y., Long, M., Chen, K., Xing, L., Jin, R., Jordan, M. I., and Wang, J.: Skilful nowcasting of extreme precipitation with NowcastNet, Nature, 619, 526–532, https://doi.org/10.1038/s41586-023-06184-4, 2023.
Zhi, W., Feng, D., Tsai, W.-P., Sterle, G., Harpold, A., Shen, C., and Li, L.: From hydrometeorology to river water quality: Can a deep learning model predict dissolved oxygen at the continental scale?, Environ. Sci. Technol., 55, 2357–2368, https://doi.org/10.1021/acs.est.0c06783, 2021.
Zhi, W., Klingler, C., Liu, J., and Li, L.: Widespread deoxygenation in warming rivers, Nat. Clim. Chang., 1–9, https://doi.org/10.1038/s41558-023-01793-3, 2023.
Zhi, W., Baniecki, H., Liu, J., Boyer, E., Shen, C., Shenk, G., Liu, X., and Li, L.: Increasing phosphorus loss despite widespread concentration decline in US rivers, Proc. Natl. Acad. Sci., 121, e2402028121, https://doi.org/10.1073/pnas.2402028121, 2024.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W.: Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting, arXiv [preprint], https://doi.org/10.48550/arXiv.2012.07436, 28 March 2021.
Executive editor
Machine learning is used widely in hydrological research nowadays, but benchmarking them for various applications was lacking. This paper addresses the question which machine learning model to be used for which application and why.
Machine learning is used widely in hydrological research nowadays, but benchmarking them for...
Short summary
Using global and regional datasets, we compared attention-based models and Long Short-Term Memory (LSTM) models to predict hydrologic variables. Our results show LSTM models perform better in simpler tasks, whereas attention-based models perform better in complex scenarios, offering insights for improved water resource management.
Using global and regional datasets, we compared attention-based models and Long Short-Term...
