ABSTRACT
Let {Xn, n ≥ 0} be a DTMC on S = {0, 1, 2, · · ·} with transition probability matrix P . In Chapter 2 we studied two main aspects of the transient behavior of the DTMC: the n-step transition probability matrix P (n) and the occupancy matrix M (n). Theorem 2.4 showed that
P (n) = Pn, n ≥ 0, and Theorem 2.6 showed that
M (n) =
P r, n ≥ 0. (4.1)
In this chapter we study the limiting behavior of P (n) as n → ∞. Since the row sums ofM (n) are n+1, we study the limiting behavior ofM (n)/(n+1) as n→∞. Note that [M (n)]ij/(n+1) can be interpreted as the fraction of the time spent by the DTMC in state j starting from state i during {0, 1, · · · , n}. Hence studying this limit makes practical sense. We begin by some examples illustrating the types of limiting
behavior that can arise.
