ABSTRACT
This chapter introduces fuzzy Q-iteration, an algorithm for approximate value iteration that relies on a fuzzy representation of the Q-function. This representation combines a fuzzy partition defined over the state space with a discretization of the action space. The convergence and consistency of fuzzy Q-iteration are analyzed. As an alternative to designing the membership functions for the fuzzy partition in advance, a technique to optimize the membership functions using the cross-entropy method is described. The performance of fuzzy Q-iteration is evaluated in an extensive experimental study.
