ABSTRACT
This chapter summarizes the basic probability theory which is an important component. It introduces delta function, Kullback-Leibler distance and its mathematical property. The chapter describes the definitions of the probability space and the random variable. Any probability space can be made complete by extending the sigma algebra and the probability measure so that any subset contained in a measure zero set belongs to the extended algebra. In probability theory, many theorems between random variables are derived. The chapter explains empirical process theory which is necessary in statistics. It discusses an infinite mixture using Dirichlet process and the finite mixture of normal distributions.
