B Efficient Neural Network Training Algorithms

The standard backpropagation algorithm introduced in Chapter 6 is notoriously slow to converge. In this appendix we will develop two additional training algorithms for the two-layer, feed-forward neural network of Figure 6.10. The first of these, scaled conjugate gradient, makes use of the second derivatives of the cost function with respect to the synaptic weights, i.e., of the Hessian matrix. The second, the extended Kalman filter method, takes advantage of the statistical properties of the weight parameters themselves. Both techniques are considerably more efficient than backpropagation.