ABSTRACT
Assuming that the above processes are irreducible (and aperiodic in the discrete time case) , we know that the steady state probabilities Xi exist (although they are positive only when the process is nonnull recurrent) . A key problem for a Markov process being the determination of its steady state probabilities, let us begin by considering the vector x = (x o , x l > . . . ). Since we know that the steady state probabilities do not depend on the initial state, we may as well assume that the process starts in state 0 at time O.
