ABSTRACT
The characteristics of neural network models are discussed, including a four-parameter generic activation function and an associated generic output function. Both supervised and unsupervised learning rules are described, including the Hebbian rule (in various forms), the perceptron rule, the delta and generalized delta rules, competitive rules, and the Klopf drive reinforcement rule. Methods of accelerating neural network training are described within the context of a multilayer feedforward network model, including some implementation details. These methods are primarily based upon an unconstrained optimization framework which utilizes gradient, conjugate gradient, and quasi-Newton methods (to determine the improvement directions), combined with adaptive steplength computation (to determine the learning rates). Bounded weight and bias methods are also discussed. The importance of properly selecting and preprocessing neural network training data is addressed. Some techniques for measuring and improving network generalization are presented, including cross validation, training set selection, adding noise to the training data, and the pruning of weights.
