ABSTRACT
This chapter begins the formal approach to using Bayesian methods of making inferences for stochastic processes, in particular, those processes with a countable number of states and an index over the set of nonnegative integers. Bayesian methods of inference consist of estimation, testing hypotheses, and making predictions. The chapter continues with the definition of a regular transition matrix and finding the stationary distribution of such chains. In order to fully understand the long-term behavior of a chain, one must know how often the states of a chain are visited, and this in turn relates to the idea of a communicating class, the ideas of recurrent and transient states, and, finally, the idea of an irreducible chain. In order to understand the idea of a recurrent and transient state, the associated graph showing the transition probabilities of the chain provides an invaluable tool.
