ABSTRACT

Sometimes an unexpected observation leads to a new way to approach an old problem. In January 1987, as part of a study of the backpropagation learning procedure, I was trying to train an artificial neural network to add a pair of vectors. This is the kind of computation that must be done in the brain to get the true spatial location of a viewed object using information about retinal location and eye position. Vector addition is linear, and, even with the nonlinear retinal representation of one of the input vectors I was using, it can be accurately approximated by a single layer of linear units. However, I used a more elaborate network than the task required; a network with an extra, or hidden, layer. This network easily learned the task; what was unexpected were the properties developed by the hidden units. The simulated retinal receptive fields and eye position responses of these units closely resembled those found in a cortical area that computes spatial location. In fact, using Richard Andersen's extensive experimental data on the response patterns of parietal neurons, it was found that the hidden units could account in considerable detail for the properties of about half the neurons in this area (Zipser & Andersen, 1987, 1988a, 1988b). This was unexpected because parietal neurons have complex response patterns that had previously been assumed to be producible only by networks that closely resembled the physiology of real cortex. These observations seem to imply that we can discover the internal representations used in the brain by using a technique that does not depend on a detailed analysis of the underlying physiology.