ABSTRACT
This chapter attempts to meet uncertainty head-on by explicitly modeling it and reasoning about it. It uses the term decision theoretic planning to refer to this broad class of planning methods characterized by explicit accounting for uncertainty. The chapter considers a number of formulations for the problem of planning under uncertainty and present algorithms for planning under these formulations. Consequently, selecting an uncertainty model can sometimes be more of an art than a science. Most of the algorithms we will present are essentially independent of uncertainty model in the sense that they can be adapted to the type of uncertainty we select. Generally, we will derive similar but distinct versions for these two uncertainty models. Many decision-theoretic planning algorithms are easiest to understand and implement under the assumption the spaces of states, actions, and observations are finite, or at least countable.
