ABSTRACT
This chapter and the next chapter are essential for understanding the content of this book. By design, the chapters present statistics from a quite different perspective as usually the statistics is introduced and taught. The theory of statistics is presented from the pure Bayesian perspective where we attempt to make sure that concepts of the classical statistics are not mixed in the exposition of the theory. In our experience, the Bayesian statistics is frequently introduced in image and signal analysis texts as an extension of the classical treatment of probability. The classical treatment of probability is based on the interpretation of probability as the frequency of occurring of some phenomena based on repeated identical trials. The classical approach is often referred to as the frequentist statistics. From the Bayesian point of view, the probability describes the strength of beliefs in some propositions. One of the most frequently used terms, the probability distribution, in frequentist statistics means the “histogram” of outcomes of the infinite number of repetitions of some experiment. In Bayesian statistics, the probability distribution quantifies beliefs or in other words measure of uncertainty. Unfortunately, these two concepts of probability, Bayesian and frequentist, are not compatible and cannot be used together in a logically coherent way. What creates confusion is that both approaches are described mathematically by the probability calculus and because of that they can be intermingled and used together which, to us at least, is incomprehensible.
