ABSTRACT
The use of neural networks for supervised learning of predetermined classifications dates back to the early work of F. Rosenblatt. The setting of the learning rate is considered in D. E. Rumelhart et al. and J. L. McClelland and Rumelhart. To make the learning of a pattern independent of its location in the visual field, all hidden units are constrained to learn exactly the same pattern of weights. Hence, living neural systems seem to include learning modules that find without direct supervision the naturally recurring input classes in the environment, but then are subject to attentional control. In reality, human and animal nervous systems probably use a mixture of localist and distributed codes. In evolution, the design of functional architectures is opportunistic, and “one size fits all” is rarely the rule. For understanding the parallel distributed processing (PDP) modeling approach, the word “distributed” is key. PDP models tend to start with units that represent broad classes of entities.
