ABSTRACT
We give a unified treatment of gradient descent learning algorithms for neural networks using a general framework of dynamical systems. This general approach organizes and simplifies all the known algorithms and results which have been originally derived for different problems (fixed point/trajectory learning), for different models (discrete/continuous), for different architectures (forward/recurrent) and using different techniques (back-propagation, variational calculus, adjoint methods, …). The general approach can also be applied to derive new algorithms. We then briefly examine some of the complexity issues and limitations intrinsic to gradient descent learning. Throughout this chapter we focus on the problem of trajectory learning.
