ABSTRACT
In some ways, the most revealing figure in Ferster and Skinner’s seminal book is their schematic showing how a pigeon learns a fixed-interval schedule. It shows the transition from a simple law-of-effect process—a burst of pecking after each reinforcement—to the fixed-interval “scallop”: a wait followed by accelerating pecking until the next food. The picture tells us that learning involves more than one process. Psychologists (behaviorists too!) want a single quantitative law to describe it all—“physics envy”). But adaptive behavior is about variation—an “adaptive toolbox,” many laws/models, not one. Recall that credit assignment seems to involve a number of systems in parallel that are selectively strengthened by contiguity with a reinforcer. That’s a prototype for how learning works. A multiplicity of systems running more or less in parallel—the variation of the Darwinian metaphor—that are selected from cues like contiguity. Sometimes the available variation allows for behavior that maximizes payoff. But sometimes variation is insufficient, or contiguity is an inadequate cue (as in any situation involving delayed reward), and the organism fails to do as well as possible. The organism is not “rational” in the first case and “irrational” in the second. Once we really understand what is happening, “rationality” plays no part in the account.
