ABSTRACT
This chapter introduces several concrete examples of statistical models. It describes normal distribution, multinomial distribution, linear regression, neural network, finite normal mixture, and nonparametric mixture. The chapter examines the behaviors of the free energy or the minus log marginal likelihood, and the generalization, training, cross validation losses, and widely applicable information criterion (WAIC). In many statistical models, the posterior average cannot be calculated analytically; hence the Markov chain Monte Carlo method is necessary. Moreover, in statistical models which have hierarchical structures or hidden variables, the posterior distribution cannot be approximated by any normal distribution. An example of such statistical model is an artificial neural network.
