EM-Based Hyperparameter Optimization for Regularized Volterra Kernel Estimation

摘要

In nonlinear system identification, Volterra kernel estimation based on regularized least squares can be performed by taking a Bayesian approach. In this framework, covariance structures which describe the Gaussian kernels are represented by a set of hyperparameters. The hyperparameters are traditionally tuned through a global optimization which maximizes their marginal likelihood with respect to the measured data. The global optimization is computationally intensive for high-order estimates, as the number of hyperparameters increases quadratically with the Volterra series order. In this letter, we propose a new method of hyperparameter tuning based on expectation-maximization (EM). The technique allows the global optimization to be split into smaller components such that the search space of any given optimization problem is not prohibitively large. The main advantage of the proposed EM method is improved computation time scaling with respect to Volterra series order. The computation time benefits of the EM-based method are demonstrated through a numerical example for the case where the maximum nonlinear order is known.