Bayesian Regularization for System Identification, Uncertainty Quantification and Model Selection

We implement a Bayesian sparse regression framework for the identification of a dynamical system from its time-series data, or more generally for model inversion from time- or spatial-series data. This is shown to provide a rationale justification for regularization methods, with the likelihood and prior corresponding respectively to the residual and regularization terms. It also provides the full Bayesian inversion apparatus (posterior and evidence), including the quantification of uncertainties in model parameters from the posterior, the calculation of regularization and nuisance parameters by explicit or iterative methods, and model selection based on the Bayesian odds ratio and evidence. A Gaussian formulation is adopted for the prior and likelihood functions, which gives analytical solutions for the posterior and evidence. The Gaussian formulation also provides “Gaussian norms” of the form ||x-µ||²_A^-1, equivalent to the square of the Mahalanobis distance between a point x and a multivariate normal distribution N(µ, A). The Gaussian norms for the posterior and evidence are shown to provide robust tools for quantitative model selection. The Bayesian framework is applied to a number of dynamical and hydrological systems.