ABSTRACT
A stochastic process is a random process evolving with time. More precisely, a stochastic process is a collection of random variables Xt indexed by time. In this book, time will always be either a subset of the nonnegative integers {0, 1, 2, . . . } or a subset of [0,∞), the nonnegative real numbers. In the first case we will call the process discrete time, and in the second case continuous time. The random variables Xt will take values in a set that we call the state space. We will consider cases both where the state space is discrete, i.e., a finite or countably infinite set, and cases where the state space is continuous, e.g., the real numbers R or d-dimensional space Rd.
