ABSTRACT
Recall that our goal is to train a two-layer net in toto without the awkwardness incurred by having to train the intermediate layer separately and hand craft the output layer. This approach also required prior knowledge about the way the training patterns fell in pattern space. A training rule was introduced-the delta rule-which, it was claimed, could be generalized from the single-unit/layer case to multilayer nets. Our approach is to try and understand the principles at work here, rather than simply use the brute force of the mathematics, and many of the fundamental ideas have already been introduced in the last chapter. A full derivation may be found, for example, in Haykin (1994). The training algorithm to be developed is called backpropagation because, as will be shown, it relies on signalling errors backwards (from output to input nodes) through the net.
