ABSTRACT

Traditionally, statisticians have modeled phenomena by supposing there are popula-

tion parameters which are fixed at certain unknown values, and then tried to estimate

(i.e. give a best guess of) these parameters. In this framework, known as the frequen-

tist framework, you observe data given that the parameter value takes some unknown

fixed value. In contrast, Bayesian statisticians treat all quantities, data and param-

eters, as random variables. In the Bayesian framework, we envision the following

process as responsible for the data we observe: the parameter values are drawn from

a probability distribution, then we observe the data conditional on the values of the

parameters. The problem of statistical inference is how to convert the observations

we have conditional on the parameter values into information about the parameters.

Bayes theorem provides a mathematically rigorous way to convert from statements

about the distribution of the data given the parameter values into statements about

the parameter values conditional on the data. Let θ represent a parameter (or set of parameters) in a statistical model (for example the mean of a distribution) and let y represent the data we have observed (in general y may be a vector, i.e. a collection of measurements). We denote the density of θ by p(θ) and the conditional distribution of θ given y by p(θ|y). This notation deviates from the common practice of denoting the density of a random variable by expressions like pθ(y) where y is the argument of the density. For example, the density of a normal distribution with mean θ and variance 1 in the latter notation is pθ(y) =