ABSTRACT
Consider a system that is modeled by a discrete-time stochastic process {Xn, n ≥ 0} with a countable state-space S, say {0, 1, 2, · · ·}. Consider a fixed value of n that we shall call “the present time” or just the “present.” Then Xn is called the present (state) of the system, {X0, X1, · · · , Xn−1} is called the past of the system, and {Xn+1, Xn+2, · · ·} is called the future of the system. IfXn = i andXn+1 = j, we say that the system has jumped (or made a transition) from state i to state j from time n to n+ 1.
