ABSTRACT
One of the goals of this book is to raise awareness that neural activity affects how different modal combinations may work best for certain purposes and certain audiences or individuals. Studies like Moreno and Mayer’s (2000) brought attention to the issue of optimal modal combinations for learning, but we do not know why those combinations worked well in that particular set of experiments. Neuroscience is the study of how neurons in various parts of the brain act under various stimuli. Cognitive neuroscience focuses study on these actions relative to learning. That is, it considers how the brain responds to various stimuli when the brain is tasked with a learning activity. The field of cognitive neuroscience integrates several fields, including cognitive psychology, philosophy, chemistry, and biology. In this chapter, I discuss the link between multimodal rhetorical theory and neurobiological scholarship in cognitive neuroscience toward identifying why multimodal materials facilitate learning. I also hope to encourage scholarship that links these fields further; I identify potential avenues for studying this link at the end of this chapter as well as in the book’s conclusion. In this chapter, in particular, I review scholarship related to cognitive neuroscience and its multi - sensory attributes. I eliminate biological jargon to focus attention on the general link between the fields of neuroscience and multimodal rhetoric. This information sheds light on why people tend to learn better when multiple modes are used to present information. Limiting the terminology to a small set of new vocabulary will also help transition rhetoricians to the science-oriented language. Little scholarship in multimodal rhetoric uses the scholarship from the field of cognitive neuroscience. One reason for this is that the biology associated with neuroscience quickly becomes too complex for most people to understand. It includes thick discussions of neurons and how information travels around the brain, integrating very specialized language associated with the biology of the brain. I try to describe some related scientific concepts without using that jargon. The scientific fields-physics, chemistry, biology, and related subfields-tend to focus study on neuron behaviors. Neurons are often considered as a network of
information-processing paths. More specifically, neurons have three principle parts: axons, dendrites, and cell body (Figure 2.1). The junction between two neurons is called the synapse. Dendrites take sensory information in, and axons send information out. The neural process involves a signal being sent from the axon to the dendrite via synapses. However, the brain develops new neurons in developmental stages of life, and it kills off neurons as we mature and grow old. Functional magnetic resonance imaging (fMRI) is one of the newest technologies used to observe neural activity in the brain as one performs a task. In particular, as one performs a given task or acquires information, certain parts of the brain are shown in fMRI scans to be more active than other areas of the brain relative to blood flow. One of the more interesting, novel studies of blood flow and task process was the study of blood flow as one views pornography. Huynh et al. (2012) found that blood flow to the visual cortex while viewing pornography (high-intensity, visual erotic stimuli) is less than normal relative to neutral visual stimuli, however, blood flow to other parts of the brain associated with sexual performance/arousal were activated. Analysis of hemodynamics and electrophysiology pertaining to neurons helps researchers understand which parts of the brain are working most during certain kinds of activities. Different cortices are associated with different functions. Figure 2.2 shows the four cortices (or lobes) of the brain. The frontal lobe is generally associated with language skills, reasoning, high-level cognition, and motor skills. The parietal lobe is associated with touch (hapo-stimuli) and memory management of other senses. If damage to this lobe
occurs, one can have difficulty, for example, with recalling how to speak cor - rectly. The occipital lobe is associated with visual stimuli (visual cortex), and the temporal lobe is associated with sound (auditory cortex) and memory associated with certain kinds of sensory information. As more blood moves to a given cortex of the brain, the implication is that the particular neural functions of that cortex are more actively engaged than are other parts of the brain; this is referred to as the “hemo-neural hypothesis” (Moore & Cao, 2008). Tools such as fMRI help identify patterns of neural activity and blood flow. Most of the research prior to 2000 focused on particular indi - vidual modal dynamics such as how the brain operates when stimulated by a single particular mode (e.g., hearing, vision, or touch). However, advances in MRI technology allow closer examinations, and recent studies have found that more than one sense is generally engaged in various activities, but to varying degrees based on the activity. There is considerable literature in the neurobiological field of cognitive neuro - science that characterizes learning as a multisensory experience (see collections edited by Calvert et al., 2004 and Murray & Wallace, 2012). Each sense inte - grates a particular mode that has already been identified in previous chapters: touch, spatial presence, visual perception, and auditory. The various modes of repre sentation are among the attributes of cognition related to learning, and the field of neuroscience asserts that cognition is a multisensory experience, which
encourages use of multimodal instructional materials. Work by Moreno and Mayer (2000) and Mayer (2005), which I have already identified, described links between learning and multimodal instructional materials. One sense not identified with linguistics and literacy that is also included in the scope of cognitive neuroscience is smell. I integrate this sense into this discussion on a limited basis, because it contributes to experience and learning. When one watches a television show about lions or penguins or other wild animals, he or she can observe animal behaviors and, depending on the quality of the microphones used, hear the sounds the animals make as they communicate with each other. A similar experience occurs when one goes to the zoo. However, the biggest difference between watching a television show about animals and going to the zoo is that one cannot smell the animals through the television experience. One is able to smell the animals when they go to the zoo. While that part of the experience may be among the less pleasant sensory experiences, it contributes to that particular kind of experience and may be part of learning about animals. For animals, distinguishing smells is among the necessary skills for survival. Animals learn to smell other animals-predators-and use that information for survival, being more careful around or avoiding smells asso - ciated with predators. That may be less important for humans, but the workplace surroundings and smell may contribute to the learning that one experiences there. Indeed, Krishna, Elder, and Caldara (2010) linked touch and smell to consumer behaviors, indicating that smell plays a role in purchasing behavior. While smell is included in this discussion, a majority of it concerns the other modes and related senses already identified.
