ABSTRACT
Simply speaking, statistical inference based on a Gaussian process regression model can be performed in the following procedure. First of all, a prior distribution for the unknown regression function needs to be specified using a Gaussian process, namely, a Gaussian process prior should be defined. The concept of a Gaussian process prior was described in (1.11), in which a form of the covariance function k(·, ·) needed to be selected. Second, the posterior distribution of a regression curve is derived via Bayes theorem based on a multivariate Gaussian distribution with a suitable mean vector and a covariance matrix, for example, (2.7) and (2.8), after we observe a set of data. Although the posterior consistency can be achieved with suitably chosen covariance functions, as discussed in the previous chapter, under some regularity conditions, the empirical selection of covariance function and the selection of hyper-parameters in the covariance function are still important issues to be considered. Since making a suitable choice of a covariance function and its hyper-parameters can improve the prediction accuracy, this is particularly important for dealing with datasets of small and medium sample sizes.
