ABSTRACT
Many scientific problems are concerned with the long-time evolution of a system. This might be a dashpot-damper system in mechanics, an ecosystem like a bird population in biology, the world atmosphere in meteorology, a society in sociology or the whole universe in cosmology. In this section we want to study special systems which can occupy only a finite number of states and in which the transition probabilities between the different states are known; the evolution of such a system is called a Markov process or a Markov chain, so-named after Andrej Andrejevitch Markov (1856–1922). The formal definition is as follows.
