ABSTRACT
The Markov chain Monte Carlo method (MCMC) enables to numerically approximate the posterior average for an arbitrary statistical model and prior. If a posterior distribution is spread on some local parameter region, then MCMC approximation is accurate, otherwise it is still not so easy. In many important statistical models such as a normal mixture or an artificial neural network, the Bayesian inference attains much more precise estimation; hence it becomes more important to construct the MCMC algorithm which works even in singular posterior distributions. The chapter discusses the basic foundations of MCMC process and explains the metropolis method. It explores numerical approximation methods of the generalization loss and the free energy using MCMC method. In order to check how accurate MCMC approximates the posterior distribution in singular cases, the real log canonical threshold would be a good index for a given set of a true distribution, a statistical model, and a prior.
