ABSTRACT
The consequences of approximating Q-Values with function approximators are investigated. Two criteria of optimality are introduced, a global and local criterion, and the viability of each is investigated for the case of a linear combination of prechosen basis functions. It is found that optimizing the global cost function is unlikely to result, in nearly optimal policies when the set of bases is not complete. The local cost function is found to be plagued by local minima.
