ABSTRACT
Much of the theory of MDPs and their approximation does not rely on the assumption that K is finite or countable, but this question is of significant practical consequence, arising from the need to first compute Taπ, then to optimize over the control variable a∈Kx. The performance of these operations enumeratively when K is finite leads to issues of complexity, but also permits the development of general algorithms with a well defined performance, and so the approximation of continuousMDPmodels using discrete models seems a natural solution.
