ABSTRACT
Markov chains are in some sense well understood, yet the dynamic behavior of their sample paths are more varied than one-dimensional dynamics in that one-dimensional dynamics can be embedded in a Markov chain, but cannot approximate all chains. Chains provide a powerful way to describe the nature of the dynamics that generated the chain whether the dynamics were observed in data or simulated from a model. Markov chains can be constructed from ‡rst principles, that is, time series data of numerical or categorical data, or simulations of ‡rst-order linear or nonlinear autoregressive models. They also provide a way to include necessary random effects in a deterministic model while providing an analytic and simulation framework for these models. This approach to describing dynamics is also applicable to situations in which the observed series is not from a Markov chain, but there is one in the background through hidden Markov models (HMM).
