ABSTRACT
Many models in science use Markov processes, which form a very rich family of stochastic processes. Let https://www.w3.org/1998/Math/MathML"> ( X ( t ) ) t ≥ 0 https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9780429086663/acb7e1d1-9bf9-43aa-8a26-560ac513e49b/content/eq248.tif" xmlns:xlink="https://www.w3.org/1999/xlink"/> , be a stochastic process taking its values in a countable space Λ. We can imagine a particle moving randomly in Λ according to certain rules. This process is a time-homogeneous Markov chain when it possesses the following Markov property: https://www.w3.org/1998/Math/MathML"> P ( X ( t n + 1 ) = x n + 1 ∣ X ( 0 ) = x 0 , X ( t 1 ) = x 1 , ⋯ , X ( t n ) = x n ) = P ( X ( t n + 1 − t n ) = x n + 1 ∣ X ( 0 ) = x n ) , https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9780429086663/acb7e1d1-9bf9-43aa-8a26-560ac513e49b/content/eq249.tif" xmlns:xlink="https://www.w3.org/1999/xlink"/>
