ABSTRACT
In Bayesian statistics, probability describes all types of uncertainty, both through unpredictability and through imperfect knowledge. Uncertainty is described by means of probability distributions that are assigned to uncertain quantities. When estimating the values of unknown parameters, which are often properties of the population, these values will be uncertain due to lack of knowledge rather than due to random variation. In the Bayesian setting, parameters are treated in a similar fashion to all other uncertain quantities; they are treated as random variables and assigned probability distributions. As such, probability statements can be made about them, such as “an interval that has 95% probability of containing the true value” or “the probability that a null hypothesis is true is...”. This is in contrast to the frequentist approach under which statements about probability are based on the idea of repetition, e.g. “95% of all confidence intervals calculated under repeated sampling will cover the true parameter value” or “the p-value for a hypothesis test is the probability that we would observe this, or something more extreme (under the assumption that the null hypothesis is true)”. Bayesian statistics may be thought of as subjective in the sense that prior knowledge and beliefs about what we expect to see can be incorporated into the inferential process. Frequentist statistics on the other hand is based solely on observed data and is therefore referred to as objective.
