Articles | Volume 27, issue 9
https://doi.org/10.5194/hess-27-1827-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/hess-27-1827-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
A signal-processing-based interpretation of the Nash–Sutcliffe efficiency
Institute of Engineering Innovation, University of Tokyo, Tokyo 113-8656, Japan
Department of Observation and Data Assimilation Research, Meteorological Research Institute, Tsukuba, Ibaraki 305-0052, Japan
Yohei Sawada
Institute of Engineering Innovation, University of Tokyo, Tokyo 113-8656, Japan
Related authors
No articles found.
Sneha Kulkarni, Yohei Sawada, Yared Bayissa, and Brian Wardlow
Hydrol. Earth Syst. Sci., 29, 4341–4370, https://doi.org/10.5194/hess-29-4341-2025, https://doi.org/10.5194/hess-29-4341-2025, 2025
Short summary
Short summary
How drought impacts communities is complex and not yet fully understood. We examined a disaster dataset and compared various drought measures to pinpoint affected regions. Our new combined drought indicator (CDI) was found to be the most effective in identifying drought events compared to other traditional drought indices. This underscores the CDI's importance in evaluating drought risks and directing attention to the most impacted areas.
Yohei Sawada
EGUsphere, https://doi.org/10.48550/arXiv.2403.06371, https://doi.org/10.48550/arXiv.2403.06371, 2024
Preprint archived
Short summary
Short summary
It is generally difficult to control large-scale and complex systems, such as Earth systems, using small forces. In this paper, a new method to control such systems is proposed. The new method is inspired by the similarity between simulation-observation integration methods in geoscience and model predictive control theory in control engineering. The proposed method is particularly suitable to find the efficient strategies of weather modification.
Yuya Kageyama and Yohei Sawada
Hydrol. Earth Syst. Sci., 26, 4707–4720, https://doi.org/10.5194/hess-26-4707-2022, https://doi.org/10.5194/hess-26-4707-2022, 2022
Short summary
Short summary
This study explores the link between hydrometeorological droughts and their socioeconomic impact at a subnational scale based on the newly developed disaster dataset with subnational location information. Hydrometeorological drought-prone areas were generally consistent with socioeconomic drought-prone areas in the disaster dataset. Our analysis clarifies the importance of the use of subnational disaster information.
Yohei Sawada, Rin Kanai, and Hitomu Kotani
Hydrol. Earth Syst. Sci., 26, 4265–4278, https://doi.org/10.5194/hess-26-4265-2022, https://doi.org/10.5194/hess-26-4265-2022, 2022
Short summary
Short summary
Although flood early warning systems (FEWS) are promising, they inevitably issue false alarms. Many false alarms undermine the credibility of FEWS, which we call a cry wolf effect. Here, we present a simple model that can simulate the cry wolf effect. Our model implies that the cry wolf effect is important if a community is heavily protected by infrastructure and few floods occur. The cry wolf effects get more important as the natural scientific skill to predict flood events is improved.
Futo Tomizawa and Yohei Sawada
Geosci. Model Dev., 14, 5623–5635, https://doi.org/10.5194/gmd-14-5623-2021, https://doi.org/10.5194/gmd-14-5623-2021, 2021
Short summary
Short summary
A new method to predict chaotic systems from observation and process-based models is proposed by combining machine learning with data assimilation. Our method is robust to the sparsity of observation networks and can predict more accurately than a process-based model when it is biased. Our method effectively works when both observations and models are imperfect, which is often the case in geoscience. Therefore, our method is useful to solve a wide variety of prediction problems in this field.
Yohei Sawada and Risa Hanazaki
Hydrol. Earth Syst. Sci., 24, 4777–4791, https://doi.org/10.5194/hess-24-4777-2020, https://doi.org/10.5194/hess-24-4777-2020, 2020
Short summary
Short summary
In socio-hydrology, human–water interactions are investigated. Researchers have two major methodologies in socio-hydrology, namely mathematical modeling and empirical data analysis. Here we propose a new method for bringing the synergic effect of models and data to socio-hydrology. We apply sequential data assimilation, which has been widely used in geoscience, to a flood risk model to analyze the human–flood interactions by model–data integration.
Cited articles
Andersson, J. C. M., Arheimer, B., Traoré, F., Gustafsson, D., and Ali, A.: Process refinements improve a hydrological model concept applied to the Niger River basin, Hydrol. Process., 31, 4540–4554, https://doi.org/10.1002/hyp.11376, 2017.
ASCE: Criteria for evaluation of watershed models, J. Irrig. Drain. Eng., 119, 429–442, 1993.
Beven, K. J. and Binley, A. M.: The future of distributed models: Model calibration and uncertainty prediction, Hydrol. Process., 6, 279–298, 1992.
Bishop, C. M.: Pattern Recognition and Machine Learning, Springer, New York, ISBN 978-0-387-31073-2, 2006.
Ding, J. Y.: Variable unit hydrograph, J. Hydrol., 22, 53–69, 1974.
Duc, L.: leducvn/gnse: gnse (v1.0), Zenodo [code], https://doi.org/10.5281/zenodo.7900649, 2023.
Fowler, K., Coxon, G., Freer, J., Peel, M., Wagener, T., Western, A., Woods, R., and Zhang, L.: Simulating Runoff Under Changing Climatic Conditions: A Framework for Model Improvement, Water Resour. Res., 54, 9812–9832, https://doi.org/10.1029/2018WR023989, 2018.
Gupta, H. V. and Kling, H.: On typical range, sensitivity and normalization of mean squared error and Nash-Sutcliffe efficiency type metrics, Water Resour. Res., 47, W10601, https://doi.org/10.1029/2011WR010962, 2011.
Knoben, W. J. M., Freer, J. E., and Woods, R. A.: Technical note: Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling–Gupta efficiency scores, Hydrol. Earth Syst. Sci., 23, 4323–4331, https://doi.org/10.5194/hess-23-4323-2019, 2019.
Lamontagne, J. R., Barber, C. A., and Vogel, R. M.: Improved estimators of model performance efficiency for skewed hydrologic data, Water Resour. Res., 56, e2020WR027101, https://doi.org/10.1029/2020WR027101, 2020.
Legates, D. R. and McCabe, G. J.: Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation, Water Resour. Res., 35, 233–241, https://doi.org/10.1029/1998WR900018, 1999.
Legates, D. R. and McCabe, G. J.: Short communication a refined index of model performance. A rejoinder, Int. J. Climatol., 33, 1053–1056, https://doi.org/10.1002/joc.3487, 2012.
Mantovan, P. and Todini, E.: Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology, J. Hydrol., 330, 368–381, 2006.
Moriasi, D. N., Arnold, J. G., Van Liew, M. W., Bingner, R. L., Harmel, R. D., and Veith, T. L.: Model evaluation guidelines for systematic quantification of accuracy in watershed simulations, T. ASABE, 50, 885–900, https://doi.org/10.13031/2013.23153, 2007.
Murphy, A.: Skill scores based on the mean square error and their relationships to the correlation coefficient, Mon. Weather Rev., 116, 2417–2424, https://doi.org/10.1175/1520-0493(1988)116<2417:SSBOTM>2.0.CO;2, 1988.
Murphy, A. H., Brown, B. G., and Chen, Y.-S.: Diagnostic verification of temperature forecasts, Weather Forecast., 4, 485–501, 1989.
Nash, J. E. and Sutcliffe, J. V.: River flow forecasting through conceptual models. Part 1: A discussion of principles, J. Hydrol., 10, 282–290, https://doi.org/10.1016/0022-1694(70)90255-6, 1970.
Pushpalatha, R., Perrin, C., Le Moine, N., and Andreassian, V.: A review of efficiency criteria suitable for evaluating low-flow simulations, J. Hydrol., 420, 171–182, 2012.
Ritter, A. and Munoz-Carpena, R.: Performance evaluation of hydrological models: Statistical significance for reducing subjectivity in goodness-of-fit assessments, J. Hydrol., 480, 33–45, https://doi.org/10.1016/j.jhydrol.2012.12.004, 2013.
Rogelis, M. C., Werner, M., Obregón, N., and Wright, N.: Hydrological model assessment for flood early warning in a tropical high mountain basin, Hydrol. Earth Syst. Sci. Discuss. [preprint], https://doi.org/10.5194/hess-2016-30, 2016.
Schaefli, B. and Gupta, H. V.: Do Nash values have value?, Hydrol. Process., 21, 2075–2080, https://doi.org/10.1002/hyp.6825, 2007.
Seibert, J.: On the need for benchmarks in hydrological modelling, Hydrol. Process., 15, 1063–1064, https://doi.org/10.1002/hyp.446, 2001.
Siqueira, V. A., Paiva, R. C. D., Fleischmann, A. S., Fan, F. M., Ruhoff, A. L., Pontes, P. R. M., Paris, A., Calmant, S., and Collischonn, W.: Toward continental hydrologic–hydrodynamic modeling in South America, Hydrol. Earth Syst. Sci., 22, 4815–4842, https://doi.org/10.5194/hess-22-4815-2018, 2018.
Stedinger, J. R., Vogel, R. M., Lee, S. U., and Batchelder, R.: Appraisal of Generalized Likelihood Uncertainty Estimation (GLUE) Methodology, Water Resour. Res., 44, WOOB06, https://doi.org/10.1029/2008WR006822, 2008.
Sutanudjaja, E. H., van Beek, R., Wanders, N., Wada, Y., Bosmans, J. H. C., Drost, N., van der Ent, R. J., de Graaf, I. E. M., Hoch, J. M., de Jong, K., Karssenberg, D., López López, P., Peßenteiner, S., Schmitz, O., Straatsma, M. W., Vannametee, E., Wisser, D., and Bierkens, M. F. P.: PCR-GLOBWB 2: a 5 arcmin global hydrological and water resources model, Geosci. Model Dev., 11, 2429–2453, https://doi.org/10.5194/gmd-11-2429-2018, 2018.
Todini, E. and Biondi, D.: Calibration, parameter estimation, uncertainty, data assimilation, sensitivity analysis, and validation, in: Handbook of applied hydrology, McGraw Hill, New York, 22-1–22-19, ISBN 9780071835091, 2017.
Towner, J., Cloke, H. L., Zsoter, E., Flamig, Z., Hoch, J. M., Bazo, J., Coughlan de Perez, E., and Stephens, E. M.: Assessing the performance of global hydrological models for capturing peak river flows in the Amazon basin, Hydrol. Earth Syst. Sci., 23, 3057–3080, https://doi.org/10.5194/hess-23-3057-2019, 2019.
Vrugt, J. A., ter Braak, C. J. F., Gupta, H. V., and Robinson, B. A.: Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?, Stoch. Env. Res. Risk A., 23, 1011–1026, 2008.
Yassin, F., Razavi, S., Elshamy, M., Davison, B., Sapriza-Azuri, G., and Wheater, H.: Representation and improved parameterization of reservoir operation in hydrological and land-surface models, Hydrol. Earth Syst. Sci., 23, 3735–3764, https://doi.org/10.5194/hess-23-3735-2019, 2019.
Wang, Z. and Bovik, A. C.: Mean squared error: Love it or leave it? A new look at signal fidelity measures, IEEE Signal Proc. Mag., 26, 98–117, https://doi.org/10.1109/msp.2008.930649, 2009.
Short summary
The Nash–Sutcliffe efficiency (NSE) is a widely used score in hydrology, but it is not common in the other environmental sciences. One of the reasons for its unpopularity is that its scientific meaning is somehow unclear in the literature. This study attempts to establish a solid foundation for NSE from the viewpoint of signal progressing. This approach is shown to yield profound explanations to many open problems related to NSE. A generalized NSE that can be used in general cases is proposed.
The Nash–Sutcliffe efficiency (NSE) is a widely used score in hydrology, but it is not common in...