ABSTRACT
This chapter introduces basic concepts in Bayesian statistics. It presents a statistical model and a prior, though a true distribution is unknown. Hence evaluation of a statistical model and a prior is necessary. The chapter provides several examples of probability distributions and illustrates two examples of posterior distributions. In a simple estimation problem, the posterior distribution can be approximated by a normal distribution, whereas in a complex or hierarchical model, the result is far from any normal distribution. The generalization loss is estimated by the cross validation loss and the widely applicable information criterion (WAIC). The chapter discusses the marginal likelihood and the free energy of statistical estimation. It examines statistical estimation in a conditional independent case, in which the cross validation loss cannot be used but WAIC can be.
