Integrating what & how/where with instrumental and Pavlovian learning: A biologically based computational model

Skill learning and instrumental conditioning have a long association with the behaviorist research program and are often thought of in terms of rote learning of simple stimulus-response (S-R) associations based on external rewards. These forms of learning have also been strongly associated with the striatum of the basal ganglia, with different striatal regions apparently making various different contributions (Atallah, Frank, & O’Reilly, 2004; Featherstone & McDonald, 2004; Graybiel, 1998; Kantak, Green-Jordan, Valencia, Kremin, & Eichenbaum, 2001; O’Doherty, 2004; Packard, Hirsh, & White, 1989; Packard & McGaugh, 1992; Poldrack, Prabhakaran, Seger, & Gabrieli, 1999; Williams & Eskandar, 2006). In the influential multiple memory systems framework of Squire (1992) for example, the basal ganglia are associated with procedural “habit” learning. However, both aspects of this dogma are currently being challenged: in the early phase of learning instrumental conditioning actually requires considerable high-level cognitive function to discover which actions lead to reward delivery in novel situations, and this early-phase high-level cognitive function depends critically on the basal ganglia, whereas the cortex is more likely the site of more rote longer-term habit learning (Atallah et al., 2004; Frank, 2005; Houk & Wise, 1995; Loh, Pasupathy, Miller, & Deco, 2008; Pasupathy & Miller, 2005). Here, we describe a biologically based computational model that demonstrates how different regions of the basal ganglia and prefrontal cortex contribute to this early learning.