ABSTRACT
This chapter examines mathematical bases of several topics in Bayesian statistics and explains the formal optimality of the Bayesian estimation. It explores a method how to construct the Bayesian hypothesis test. The chapter discusses Bayesian model comparison method which is different from the Bayesian hypothesis test and describes the concept of the phase transition of the posterior distribution. There are different cross validation losses and information criteria according to the different predictive losses. The chapter considers Bayesian model comparison using the posterior distribution. This decision rule is different from the hypothesis test. In Bayesian statistics, the following concepts are all mathematically connected. In order to analyze the phase transition, we can choose the most convenient one. The concepts are: partition function, free energy or the minus log marginal likelihood, state density function, zeta function, and posterior distribution.
