ABSTRACT

Let {Xn, n ≥ 0} be a DTMC on S = {0, 1, 2, · · ·}, with transition probability matrix P , and initial distribution a. Let

T = min{n ≥ 0 : Xn = 0}. (3.1) The random variable T is called the first passage time into state 0, since T is the first time the DTMC “passes into” state 0. Although we shall specifically concentrate on

this first passage time, the same techniques can be used to study the first passage time

into any set A ⊂ S. (See Conceptual Exercises 3.1 and 3.2.) In this chapter we shall study the following aspects of the first passage time T :

(1) Complementary cumulative distribution function of T : v(n) = P(T > n), n ≥ 0,

(2) Probability of eventually visiting state 0: u = P(T <∞), (3) Moments of T :m(k) = E(T k), k ≥ 1, (4) Generating function of T : φ(z) = E(zT ).