ABSTRACT
The initial ideas on iterative dynamic programming were developed and tested by Luus and then refined to make the computational procedure much more efficient. The fundamental ideas about using dynamic programming are based on an optimal path problem and are adapted to the solution of an optimal control problem. To set up the problem into a sequence of stages, as required for dynamic programming, the optimal control problem is approximated by requiring a piecewise constant control policy instead of a continuous control policy. This chapter discusses the algorithm for iterative dynamic programming (IDP) and early applications of IDP. IDP provides a very convenient way of investigating the effect of the choice of the final time in optimal control problems. The additional advantage of IDP is that the computations are straightforward and the algorithm can be easily programmed to run on a personal computer.
