ABSTRACT

Several recent and classical biological applications of DTMC models are discussed in this chapter. The first application is a recent application of finite Markov chain theory to proliferating epithelial cells. Then a classical biological application of a DTMC is discussed, a random walk on a finite domain, often referred to as the gambler’s ruin problem. Expressions are derived for the probability of absorption, the expected duration until absorption, and the entire probability distribution for absorption at the nth time step. Some of these results are extended to a random walk on a semi-infinite domain {0, 1, 2 . . .}. Another classical application of DTMCs is to birth and death processes. A general discrete-time birth and death process is described, where the domain for the population is {0, 1, 2, . . .}. The general birth and death process is applied to a logistic birth and death process, where the birth and death probabilities are nonlinear functions of the population size. In this model, it is assumed that there is a maximal population size, so that the process simplifies to a finite Markov chain, where a transition matrix can be defined. The theory developed from random walk models is useful to the analysis of birth and death processes. The probability of absorption or population extinction, the expected time until population extinction, and the distribution conditioned on nonextinction, known as the quasistationary distribution, are studied.