ABSTRACT
Discrete-time Markov chains are introduced in this chapter, where the time and state variables are discrete-valued. Some basic notation and theory for discrete-time Markov chains are presented in the first section. Then discretetime Markov chains are classified into one of several types, irreducible or reducible, periodic or aperiodic, and recurrent or transient. This classification scheme determines the asymptotic behavior of the Markov chain. For example, an aperiodic, irreducible, and recurrent Markov chain is shown to have a stationary limiting distribution. Some well-known examples of discrete-time Markov chains are discussed, including the random walk model in one, two, and three dimensions and a random walk model on a finite domain, often referred to as the gambler’s ruin problem. Another biological example that is discussed is the genetics of inbreeding.
