ABSTRACT
Markov chains are encountered in several applications arising in different contexts, and model many real problems which evolve in time. Throughout, we denote by P[X = j] the probability that the random variable X takes the value j, and by P[X = j |Y = i] the conditional probability that X takes the value j, given that the random variable Y takes the value i. Moreover, we denote by E[X] the expected value of the random variable X and by E[X| A] the conditional expectation of X, given the event A.
