Foundations of Bayesian Inference and Application to Dynamical System Identification

This opening double-length oral presentation of this minisymposium first examines the foundations of Bayesian inference, and its breadth and scope for all inference problems in all branches of science and engineering. This includes an overview of different schools of probability, the role of deductive vs plausible reasoning, the interpretation of probabilities as “plausibilities”, the basis of Bayes’ theorem, and the meaning of its key constructs (prior, likelihood, posterior and evidence). These are illustrated by several simple examples.

The presentation then examines the solution of inverse problems, involving the identification of a model from its data. This is demonstrated by the identification of a dynamical system from time-series data, using a Bayesian framework. This is conducted by the maximum a posteriori (MAP) point estimate, which is shown to give a generalized Tikhonov regularization method, in which the residual term corresponds to the likelihood and the regularization term corresponds to the prior. Although this provides a point estimate, the Bayesian interpretation provides access to the full Bayesian apparatus, including the ranking of models, the quantification of model uncertainties, the estimation of unknown (nuisance) hyperparameters, and (if desired) exploration of the model space. Two Bayesian algorithms for hyperparameter estimation – the joint maximum a posteriori (JMAP) method and the variational Bayesian approximation (VBA) – are applied to several dynamical systems with additive noise, in comparison to several sparse regression algorithms including SINDy, LASSO and ridge regression. The advantages of the Bayesian framework for model selection and uncertainty quantification are demonstrated clearly.