ABSTRACT
In this chapter, the authors introduce terminology for describing the various characteristics of a Markov chain. They consider a vastly larger example of a stationary distribution, for a Markov chain on a state space with billions of interconnected nodes: the World Wide Web. Markov chains were first introduced in 1906 by Andrey Markov, with the goal of showing that the law of large numbers can apply to random variables that are not independent. Thus Markov chains are a happy medium between complete independence and complete dependence. Since their invention, Markov chains have become extremely important in a huge number of fields such as biology, game theory, finance, machine learning, and statistical physics. The Markov property greatly simplifies computations of conditional probability: instead of having to condition on the entire past, the authors only need to condition on the most recent value. The transition probabilities of a Markov chain can also be represented with a diagram.
