Parameter and predictive outcomes of model simplification and calibration
The most important aspect of modelling for environmental management is accurate characterisation of the uncertainty in a given model’s predictions of future system behaviour. This allows for conservative management.
A model of an environmental system will always be a simplification, as available information is never sufficient to capture every detail of reality. Furthermore, the computational requirements of model calibration and calibration-constrained uncertainty analysis typically demand further simplification.
This work involves a systematic analysis of the effects of model simplification upon calibrated model parameters and ultimately model predictive performance. Paired model analysis and linear analysis techniques are applied to synthetic examples in order to glean an in-depth understanding of the effects of model simplification and calibration.
This work elucidates the complexities in the parameter and predictive outcomes of model simplification and calibration. It is shown that substantial bias in some model predictions may result from calibration of even very mildly simplified models, this eroding predictive uncertainty reduction gains achieved through history-matching. It is shown that calibration will reduce the potential error in some predictions regardless of how drastic the simplification and regardless of the calibration approach. For other predictions, predictive bias incurred through calibrating a simplified model may in fact be so large that potential predictive error becomes greater than if the model had not been calibrated at all. It is found that bias may be reduced through adoption of an appropriate calibration strategy. However, a strategy that reduces bias in one prediction may increase bias in another. Thus the highly prediction-specific nature of the outcomes of simplification and calibration, and thus lack of a single best-practice approach to predictive modelling, is emphasised: in some cases history-matching is at a premium, while in others calibration may in fact do more harm than good and therefore perhaps should be abandoned completely in favour of non-calibration-constrained uncertainty analyses.
Ty Watson
John Doherty; Adrian Werner; Craig Simmons