ABSTRACT
This chapter begins with laying the theoretical foundation for the study of continuous-time Markov chains (CTMC). It considers that the important concepts involved in the study of some processes and describes the ideas of time homogeneity and the probability transition function. The chapter explains the ideas of transition rates, holding times, embedded chain, and transition probabilities. One of the most important concepts for CTMC is the infinitesimal transition rate matrix, which leads to the transition probabilities of the embedded chain. Such concepts are illustrated by using R to compute the transition function and to generate observation from the CTMC. This is followed by a presentation of Bayesian inferences of estimation and testing hypotheses about the unknown parameters of the process. Also developed is the Bayesian predictive distribution for future observations of the CTMC. The chapter concludes with many examples, including deoxyribonucleic acid (DNA) evolution, birth and death processes, and queuing.
