ABSTRACT
In his famous book published a little more than half a century ago, The Organization of Behavior, Hebb proposed a hypothetical mechanism by which ‘cell assemblies’, which are a group of cells that could act like a form of ‘short term memory’ and support selfsustaining reveberatory activity outlasting the input, could be constructed [37]. These suggestions were later extended into other areas and now serve as the basis for a large body of thinking concerning activity-dependent processes in development, learning, and memory [7, 13, 28, 35, 52, 74, 100]. What Hebb proposed was an elegant way for correlated, i.e., interesting, features of an input stimulus to become permanently imprinted in the architecture of neural circuits to alter subsequent behavior, which is the hallmark of learning. It is similar in form to classical conditioning in the psychology literature. Many models have subsequently been constructed based on extensions of this simple rule, now commonly called the Hebbian rule. These models have given reasonable accounts of many aspects of development and learning [44, 48, 58, 59, 62, 70, 73, 75, 77, 90, 95, 97]. In this chapter, we will not attempt to review the literature on Hebbian learning exhaustively. Instead, we will try to review some relevant facts from the Hebbian learning literature and discuss their connections to spike-timing-dependent plasticity (STDP), which are based on recent experimental data. To discuss Hebbian learning and STDP in a coherent mathematical framework, we need to introduce some
formalism. Let us consider one neuron receiving many inputs labelled 1 to N and denote the instantaneous rate for the ith input as and the output as (rout). The integration performed by the neuron could be written as
(11.1)
where rout (t) is the instantaneous firing rate of the output neuron at time t, G is a constant
gain factor for the neuron, wi is the synaptic strength of the ith input, and is the instantaneous firing rate of the ith input at time t. Solving the differential equation, we have
(11.2)
with
(11.3)
K(t) is a kernel function used to simulate the membrane integration performed by the neuron and θ is the threshold. Therefore, the rate of a given neuron is linearly dependent on the total amount of input into the neuron over the recent past, with exponentially more emphasis on the most recent inputs. For the sake of simplicity, we do not include the rectifying nonlinearity introduced by the threshold and only consider the regime above threshold. If we assume that plasticity is on a slower time scale than changes in firing rates and only depend on the average firing rates, we can further simplify Equation (11.2) to
(11.4)
This simplification is however not appropriate when we consider plasticity rules that depend on spike time later in this chapter. Computationally, the simplest rule that follows from Hebb’s idea is probably
(11.5)
We will drop the ‘(t)’ term in subsequent equations and the reader should note that all these entities represent functions in time. After plugging in Equation (11.4), wehave
(11.6)
If we average over a long time, we can write the rule as
(11.7)
where represents the average correlation between the inputs over the training sets. If this rule is applied for a long enough time, the weight vectors would pick out the principal eigenvector of the correlation matrix of the inputs the neuron experienced [62].
