A simulation model without statistical analysis is useless (?!)

Simulation models of different types are commonly used in a wide variety of applications, such as crop yield prediction, forecasting climate change effects, quantifying effects of shocks on ecosystems or sociotechnical systems like water systems or electrical grids, or social simulation.

The utility of models for these applications depends on our ability to analyze their output, for instance, to get estimates of uncertainties in predictions, and to calibrate them to existing data. If we cannot analyze or calibrate models, we cannot validate or perhaps even understand their output, and then what is their use besides academic exercise?

For Agent Based Models and many other simulation model types there are no standardized analysis approaches. This hampers the utility of these models for engineering and policy applications. This project is aimed at the development and application of (preferably automated) statistical methodologies for model analysis and calibration, for example, the use of different types of sensitivity analysis or a sound design for numerical experiments.

Suggested starting literature:

Cariboni, J., et al. "The role of sensitivity analysis in ecological modelling." Ecological modelling 203.1-2 (2007): 167-182.

Lee, Ju Sung, et al. "The complexities of agent-based modeling output analysis." The journal of artificial societies and social simulation 18.4 (2015).

Ten Broeke, G., G. van Voorn, and A. Ligtenberg. "Which sensitivity analysis method should I use for my agent-based model?." Journal of Artificial Societies and Social Simulation 19.1 (2016).

Other people involved in the supervisoin

One or more statisticians at Biometris (Fred van Eeuwijk, Gerrit Gort, Bas Engel, Shota Gugushvili, Daniela Bustos Korts, Willem Kruijer).

Other connections

Depending on the type of model you are interested in other parties may be involved. In principle a collaboration is intended to work on a simulation model within the context of resilience of socio-ecological systems (DeSIRE). However, depending on the taste of the MSc candidate, an alternative application field can be selected. We have for example connections with companies with an interest in crop growth modelling, and institutions involved in (urban) policy making.

Used skills

Statistics, software programming, some modelling when desired


A solid statistical background (with adequate grades for statistical courses) and knowledge of software programming (like R) is highly desirable.Affinity with simulation models or particular model applications will be helpful.


At the moment (due to COVID-19) the work should probably be conducted remotely (online). For students from Leiden we strive for at least once or twice a week to be at Wageningen.