ABSTRACT
In Chapters 2, 3, and 4, the stochastic processes were discrete in time. In this chapter, continuous-time Markov chains are introduced, where time is continuous, t ∈ [0,∞), but the random variables are discrete. Notation and basic definitions are introduced and a corresponding discrete-time process is defined, known as the embedded Markov chain. As a preliminary example, one of the simplest continuous-time Markov chains, the Poisson process is defined. The Poisson process serves as an illustration for the techniques that will be used in this chapter and lays the foundation for the construction of more general birth and death and population processes discussed in later chapters.
