Forward Models: Supervised Learning with a Distal Teacher
Recent work on learning algorithms for connectionist networks has seen a pro gressive weakening of the assumptions made about the relationship between the learner and the environment. Classical supervised learning algorithms such as the perceptron (Rosenblatt, 1962) and the LMS algorithm (Widrow & Hoff, 1960) made two strong assumptions: (1) the output units are the only adaptive units in the network; and (2) there is a "teacher" that provides desired states for all of the output units. Early in the development of such algorithms it was recognized that more powerful supervised learning algorithms could be realized by weakening the first assumption and incorporating internal units that adaptively recode the input representation provided by the environment (Rosenblatt, 1962). The subse quent development of algorithms such as Boltzmann learning (Hinton & Sejnowski, 1986) and back propagation (LeCun, 1985; Parker, 1985; Rumelhart, Hin ton, & Williams, 1986; Werbos, 1974) have provided the means for training networks with adaptive nonlinear internal units. The second assumption has also
been weakened-learning algorithms that require no explicit teacher have been developed (Becker & Hinton, 1989; Grossberg, 1987; Kohonen, 1982; Linsker, 1988; Rumelhart & Zipser, 1986). Such "unsupervised" learning algorithms gen erally perform some sort of clustering or feature extraction on the input data and are based on assumptions about the statistical or topological properties of the input ensemble.