ABSTRACT
This chapter discusses some mathematical concepts and outlines some elements of probability theory. It discusses some basics of convex optimization and introduces entropy and relative entropy. The chapter reviews the formal definition of a probability space as formulated by Kolmogorov. The Gaussian distribution, also known as the normal distribution, though not suitable for all modeling purposes, is an extremely useful distribution and is perhaps the most widely used probability distribution. The practical importance and usefulness of convexity of a optimization problem is to a large extent derived from the fact that any local minimum of a convex function on a convex set is a global minimum. Many numerical algorithms for finding a global minimum of a function focus on the search for local minima, i.e., minima of the function in small neighborhoods. The chapter provides definitions for some standard terms relating to the moments of random variables.
