Preprints
https://doi.org/10.5194/hess-2023-258
https://doi.org/10.5194/hess-2023-258
09 Nov 2023
 | 09 Nov 2023
Status: a revised version of this preprint was accepted for the journal HESS and is expected to appear here in due course.

When ancient numerical demons meet physics-informed machine learning: adjoint-based gradients for implicit differentiable modeling

Yalan Song, Wouter J. M. Knoben, Martyn P. Clark, Dapeng Feng, Kathryn E. Lawson, and Chaopeng Shen

Abstract. Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, has shown promise in enhancing hydrologic models’ accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur large numerical errors whose impacts on models’ representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a “discretize-then-optimize” adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrologic modeling. The adjoint model demonstrates comprehensively improved performance, with Kling-Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model’s ability to represent hydrologic dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low and peak flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model’s success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Yalan Song, Wouter J. M. Knoben, Martyn P. Clark, Dapeng Feng, Kathryn E. Lawson, and Chaopeng Shen

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on hess-2023-258', Ilhan Özgen-Xian, 09 Dec 2023
    • AC1: 'Reply on RC1', Chaopeng Shen, 07 Jan 2024
  • RC2: 'Comment on hess-2023-258', Uwe Ehret, 18 Dec 2023
    • AC2: 'Reply on RC2', Chaopeng Shen, 07 Jan 2024
    • AC3: 'Reply on RC2', Chaopeng Shen, 12 Jan 2024
      • RC3: 'Reply on AC3', Uwe Ehret, 17 Jan 2024
        • AC4: 'Reply on RC3', Chaopeng Shen, 03 Feb 2024

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on hess-2023-258', Ilhan Özgen-Xian, 09 Dec 2023
    • AC1: 'Reply on RC1', Chaopeng Shen, 07 Jan 2024
  • RC2: 'Comment on hess-2023-258', Uwe Ehret, 18 Dec 2023
    • AC2: 'Reply on RC2', Chaopeng Shen, 07 Jan 2024
    • AC3: 'Reply on RC2', Chaopeng Shen, 12 Jan 2024
      • RC3: 'Reply on AC3', Uwe Ehret, 17 Jan 2024
        • AC4: 'Reply on RC3', Chaopeng Shen, 03 Feb 2024
Yalan Song, Wouter J. M. Knoben, Martyn P. Clark, Dapeng Feng, Kathryn E. Lawson, and Chaopeng Shen
Yalan Song, Wouter J. M. Knoben, Martyn P. Clark, Dapeng Feng, Kathryn E. Lawson, and Chaopeng Shen

Viewed

Total article views: 1,317 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
949 333 35 1,317 28 35
  • HTML: 949
  • PDF: 333
  • XML: 35
  • Total: 1,317
  • BibTeX: 28
  • EndNote: 35
Views and downloads (calculated since 09 Nov 2023)
Cumulative views and downloads (calculated since 09 Nov 2023)

Viewed (geographical distribution)

Total article views: 1,278 (including HTML, PDF, and XML) Thereof 1,278 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 30 May 2024
Download
Short summary
Wouldn't it be nice to have both the accuracy of neural networks (NNs) and the interpretability of process-based models (PBMs)? Differentiable modeling gives you the best of both worlds by connecting NNs with PBMs. However, there was previously a major issue that iterative solution schemes would run into memory use trouble. This paper presents an operator called adjoint, which liberates all the iterative solvers. This is the first time adjoint is applied to large-scale hydrologic modeling.