Model assessment in scientific computing: Considering robustness to uncertainty in input parameters
Abstract
Purpose
This paper aims to focus on the assessment of the ability of computer models with imperfect functional forms and uncertain input parameters to represent reality.
Design/methodology/approach
In this assessment, both the agreement between a model’s predictions and available experiments and the robustness of this agreement to uncertainty have been evaluated. The concept of satisfying boundaries to represent input parameter sets that yield model predictions with acceptable fidelity to observed experiments has been introduced.
Findings
Satisfying boundaries provide several useful indicators for model assessment, and when calculated for varying fidelity thresholds and input parameter uncertainties, reveal the trade-off between the robustness to uncertainty in model parameters, the threshold for satisfactory fidelity and the probability of satisfying the given fidelity threshold. Using a controlled case-study example, important modeling decisions such as acceptable level of uncertainty, fidelity requirements and resource allocation for additional experiments are shown.
Originality/value
Traditional methods of model assessment are solely based on fidelity to experiments, leading to a single parameter set that is considered fidelity-optimal, which essentially represents the values which yield the optimal compensation between various sources of errors and uncertainties. Rather than maximizing fidelity, this study advocates for basing model assessment on the model’s ability to satisfy a required fidelity (or error tolerance). Evaluating the trade-off between error tolerance, parameter uncertainty and probability of satisfying this predefined error threshold provides us with a powerful tool for model assessment and resource allocation.
Keywords
Acknowledgements
The authors thank Parker Shields, Ismail Farajpour and Greg Roche for their contribution during the earlier stages of this work, as well as Godfrey Kimball for his editorial help.
Citation
Prabhu, S., Atamturktur, S. and Cogan, S. (2017), "Model assessment in scientific computing: Considering robustness to uncertainty in input parameters", Engineering Computations, Vol. 34 No. 5, pp. 1700-1723. https://doi.org/10.1108/EC-03-2016-0109
Publisher
:Emerald Publishing Limited
Copyright © 2017, Emerald Publishing Limited