Journal cover Journal topic
Hydrology and Earth System Sciences An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

IF value: 5.153
IF5.153
IF 5-year value: 5.460
IF 5-year
5.460
CiteScore value: 7.8
CiteScore
7.8
SNIP value: 1.623
SNIP1.623
IPP value: 4.91
IPP4.91
SJR value: 2.092
SJR2.092
Scimago H <br class='widget-line-break'>index value: 123
Scimago H
index
123
h5-index value: 65
h5-index65
Preprints
https://doi.org/10.5194/hess-2020-128
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/hess-2020-128
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

  06 Apr 2020

06 Apr 2020

Review status
A revised version of this preprint is currently under review for the journal HESS.

Technical note: Bit by bit: A practical and general approach for evaluating model computational complexity vs. model performance

Elnaz Azmi1, Uwe Ehret2, Steven V. Weijs3, Benjamin L. Ruddell4, and Rui A. P. Perdigão5,6,7 Elnaz Azmi et al.
  • 1Steinbuch Centre for Computing, Karlsruhe Institute of Technology – KIT, Karlsruhe, Germany
  • 2Institute of Water Resources and River Basin Management, Karlsruhe Institute of Technology – KIT, Karlsruhe, Germany
  • 3Department of Civil Engineering, University of British Columbia, Canada
  • 4School of Informatics, Computing, and Cyber Systems, Northern Arizona University, USA
  • 5Meteoceanics Interdisciplinary Centre for Complex System Science, Vienna, Austria
  • 6CCIAM, Centre for Ecology, Evolution and Environmental Changes, Universidade de Lisboa, Lisbon, Portugal
  • 7Physics of Information and Quantum Technologies Group, Instituto de Telecomunicações, Lisbon, Portugal

Abstract. One of the main objectives of the scientific enterprise is the development of parsimonious yet well-performing models for all natural phenomena and systems. In the 21st century, scientists usually represent their models, hypotheses, and experimental observations using digital computers. Measuring performance and parsimony for computer models is therefore a key theoretical and practical challenge for 21st century science. The basic dimensions of computer model parsimony are descriptive complexity, i.e. the length of the model itself, and computational complexity, i.e. the model's effort to provide output. Descriptive complexity is related to inference quality and generality, and Occam's razor advocates minimizing this complexity. Computational complexity is a practical and economic concern for limited computing resources. Both complexities measure facets of the phenomenological or natural complexity of the process or system that is being observed, analysed and modelled.

This paper presents a practical technique for measuring the computational complexity of a digital dynamical model and its performance bit by bit. Computational complexity is measured by the average number of memory visits per simulation time step in bits, and model performance is expressed by its inverse, information loss, measured by conditional entropy of observations given the related model predictions, also in bits. We demonstrate this technique by applying it to a variety of watershed models representing a wide diversity of modelling strategies including artificial neural network, auto-regressive, simple and more advanced process-based, and both approximate and exact restatements of experimental observations. Comparing the models revealed that the auto-regressive model poses a favourable trade-off with high performance and low computational complexity, but neural networks and high-time-frequency conceptual bucket models pose an unfavourable trade-off with low performance and high computational complexity. We conclude that the bit by bit approach is a practical approach for evaluating models in terms of performance and computational complexity, both in the universal unit of bits, which also can be used to express the other main aspect of model parsimony, description length.

Elnaz Azmi et al.

Interactive discussion

Status: final response (author comments only)
Status: final response (author comments only)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Login for Authors/Editors] [Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement

Elnaz Azmi et al.

Elnaz Azmi et al.

Viewed

Total article views: 647 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
510 122 15 647 15 16
  • HTML: 510
  • PDF: 122
  • XML: 15
  • Total: 647
  • BibTeX: 15
  • EndNote: 16
Views and downloads (calculated since 06 Apr 2020)
Cumulative views and downloads (calculated since 06 Apr 2020)

Viewed (geographical distribution)

Total article views: 384 (including HTML, PDF, and XML) Thereof 383 with geography defined and 1 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 

Cited

Saved

No saved metrics found.

Discussed

No discussed metrics found.
Latest update: 19 Sep 2020
Publications Copernicus
Download
Short summary
Computer models should be as simple as possible, but not simpler. Simplicity refers to the length of the model, and the effort it takes the model to generate its output. Here we present a practical technique for measuring the latter by the number of memory visits during model execution by Strace, a troubleshooting and monitoring program. The advantage of this approach is that it can be applied to any computer-based model, which facilitates model intercomparison.
Computer models should be as simple as possible, but not simpler. Simplicity refers to the...
Citation