ABSTRACT
This chapter is a short course on probability theory, containing standard material, such as discrete probability theory, random variables, statistical averages, variances, various distributions (e.g., the Gaussian), Markov chains, correlations, etc. Some issues are emphasized. The chapter elaborates (with examples) on the related notions of experimental probability versus probability space, advocating for the latter to be used as a framework for solving probability problems in a systematic way. When analytically unsolvable problems are encountered, one has to move from the probability space to the experimental probability on a computer, that is, to computer simulation. The distinction between these two phases, which is sometime confusing to students, is essential for devising simulation methods, in particular, for the entropy and free energy. Longer discussions than usual are devoted to the concept of product space, which constitutes the theoretical framework for simulation and estimation theories; special attention is given to the central limit theorem (CLT) – a highly used tool throughout the book. Finally, a basic property related to entropy, which will be discussed later in the book, is emphasized. Thus, while dynamic simulation methods, such as Monte Carlo and molecular dynamics, sample system’s states with the Boltzmann probability, P B, the value of P B is unknown and the absolute entropy is unknown as well.
