ABSTRACT
Function approximation from a set of input-output pairs has numerous applications in the fields of signal processing, pattern recognition and computer vision. Recently, feedforward neural networks containing hidden layers have been proposed as a tool for nonlinear function approximation as in Hornik et al. (1989), Poggio and Girosi (1990), and Ji et al. (1990). Parametric models represented by such networks are a cascade of nonlinear functions of affine transformations. In particular, approximation using a three-layer network is closely related to projection-based approximation, a parametric form first used in projection pursuit regression and classification studied by Friedman and Stuetzle (1981) and Friedman (1985). Diaconis and Shahshahani (1984) have shown that the projection-based approximation is dense in any real-valued continuous function. Recently, Hornik et al. (1989) extended this result by showing that a three-layer network is actually capable of approximating any Borel measurable function, provided sufficiently many hidden layer units are available. Due to these results, multi-layer networks have been referred to as universal approximators (Hornik et al., 1989).
