ABSTRACT
This chapter introduces the concept of continuous time Markov chain, continuous time Markov chain (CTMC), by discussing the respective Chapman-Kolmogorov equation, rate matrix, holding time, and transient and steady-state solutions. This chapter examines the concepts of mean recurrence time, mean first passage time, holding time, and mean time to absorption. In CTMC, a state change may occur at any point in time. The method is named Gauss elimination since it involves combining equations to eliminate unknowns. Although it is one of the earliest methods for solving linear equations, it is still one of the most important algorithms and the basis for solving linear equations on many software tools. Iterative methods are seldom adopted for analyzing small dimension Markov chains since the time needed for reaching the required accuracy exceeds those of direct methods, such as Gaussian elimination.
