ABSTRACT
When income or costs are associated with the states of a Markov chain, the system is called a Markov chain with rewards, or MCR. This chapter, which treats an MCR, has two objectives. The fi rst is to show how to calculate the economic value of an MCR. The second is to use an MCR to link a Markov chain to a Markov decision process (MDP), thereby unifying the treatment of both subjects. In Chapter 5, an MDP, is constructed by associating decision alternatives with a set of MCRs. Thus, an MDP can be viewed simply as a set of Markov chains with rewards plus decisions.
