ABSTRACT
This chapter dedicates to the subset of machine learning that makes prior assumptions on parameters. It explains how Bayes' theorem can be applied to simple building blocks in machine learning, and introduces some notations and concepts. In Bayesian analysis, one sophistication (compared to a frequentist approach) comes from the fact that the data is not almighty. The distribution of the parameter θ will be a mix between some prior distribution set by the statistician and the empirical distribution from the data. Bayesian methods are widely used for portfolio choice. The rationale is that the distribution of asset returns depends on some parameter and the main issue is to determine the posterior distribution. The Gibbs algorithm can be considered as a particular case of the Metropolis-Hastings method, which, in its simplest version, was introduced in Metropolis and Ulam.
