ABSTRACT

This book is primarily about language. However, we live in a digital age awash in images and texts that combine words and images. Texts that combine words and images are often called ‘‘multimodal texts,’’ because they combine different modes like language, images, and music. Such things as ads, music videos, and video games are obvious examples. Even academic texts like science textbooks and professional publications today combine a great many images (e.g., pictures, graphs, and diagrams) with language. The theory of discourse analysis in this book applies, in large part, to ‘‘texts’’ which are composed of just images, and to multimodal texts composed of combinations of words, images, and other modalities, provided these images and combinations are meant to communicate. The theory applies because, in fact, discourse is about communication and we humans can communicate via other symbol systems (e.g., mathematics) or via systems composed using modalities other than language or ones composed by mixing other modalities with language. We have argued that discourse analysis starts by asking questions which are tools for doing discourse analysis. We have offered 27 such tools. Suitably adapted, all of these questions can apply to studying images and multimodal texts (except the ones germane to grammar that we developed in our Grammar Interludes, which can, however, apply to the language used in multimodal texts, if not to the images). Let me here use the word ‘‘image’’ to mean either a static image (like a painting) or moving images (as in a movie or video game). And let us, for a moment, forget about words. If we want to analyze an image (remember I am using ‘‘image’’ to mean a single static image or a set of changing or moving images), first we have to ask what constitutes a ‘‘word’’ or ‘‘phrase’’ in the image. That is, what constitutes a small unit of meaning which can be combined with other such small units to make bigger units of meaning? When you want to analyze an image, start by asking yourself what are the ‘‘elements’’ (parts) in the image out of which it seems to be

composed. Take any element that seems to be an important ‘‘part’’ out of which the whole image is composed. This will change for various images you choose and in terms of what you want to analyze. In one case, it might be colors and shapes and in another case it might be the objects that compose the whole image. There is a ‘‘grammar’’ of images in terms of which we could formalize what counts as an element and what are the ‘‘rules’’ for combining them. But we do not need to be fussy right here and now about what counts as an element. As with language, any image communicates (has meaning) only in context and leaves much ‘‘unsaid,’’ assuming it will be filled in by people’s knowledge of the context, including their cultural knowledge and former experience with such images. So our Fill In Tool (Tool 2) applies fully here. For example, when we see attractive people associated with products in an ad (something that happens often) we fill in, from our background and cultural knowledge, the idea that the ad is trying to suggest that the product will make us more attractive or, at least, will make other people see us as associated with attractive people and an attractive life style. Video game players do not need to be told to break crates and boxes in a game to look for ‘‘power ups’’ in games-they know this from their previous experience with games. Images, just like communication in language, do not just ‘‘say’’ things (carry ‘‘messages’’), but seek to do things as well. So our Doing and Not Just Saying Tool (Tool 7) applies here as well. It is clear that ads want us to do something, namely buy the product being advertised. Video games want players to do certain things (play in certain ways). Posters and documentaries often want us to change our political views or change how we act in society. Let’s consider our theoretical tools, the tools we introduced in Unit 4: situated meanings (Tool 23); social languages (Tool 24); intertextuality (Tool 25); figured worlds (Tool 26); and Discourses (Tool 27). For whatever you take to be an element, ask what situated meaning (contextually specific meaning) this element has in the context in which the image is being ‘‘read.’’ For example, the images of petals and flowers are both elements of larger images and scenes in the video game Flower. In the game-which is really a sort of poem-the petals and flowers take on a situated meaning as ‘‘forces of unspoiled nature imperiled by industry and pollution.’’ Go on from situated meanings to ask yourself how the elements you have found fit together-form a pattern-that creates a certain sort of style for the whole image. This is the equivalent of a ‘‘social language.’’ Just as wearing a swim suit, a tank-top, and flip-flops pattern together to create a ‘‘beach style,’’ so, as we have seen, certain sorts of words, phrases, and structures can pattern together to create a certain sort of style of language (e.g., hip-hop or the language of physics), what we

have called a social language. So, too, in an image, the elements can pattern together to create a certain style. For example, in many role-playing games (like Dungeons and Dragons or World of War Craft) the elements, and the images and scenes into which they combine, pattern together to betoken a fantasy medieval world (thus, we get things like elves, castles, knights, magic and mages, and so forth). This sort of medieval fantasy style (‘‘social language’’) is often associated with and in many cases inspired by Tolkien’s Lord of the Rings. Just as with social languages, different styles are associated with different identities and activities (practices). The medieval fantasy style of many role-playing games is associated with a heroic identity and the activities of heroes and warfare. People in such games take on fantasy identities as heroes (and sometimes as mightily evil beings). The elements in an image, alone or together with other elements, can make intertextual references to other images, texts, or media (just as we can do with words in language). Furthermore, just as we saw for language, we can also talk about textual mixing, where one text (here an image) mixes elements from different styles or sources. In fact, in modern media, intertextuality and mixing (allusions to other texts and media, mixing, and remixing of styles and sources) are often pervasive. For example, later I will analyze a television commercial for Hummers that uses characters that look and act like they are in an old-fashioned Japanese horror movie (where giant monsters attack cities). The commercial is referencing such movies to sell cars. Many anime films so mix Japanese and Western cultural values and images that it is better to see them as mixing these two than simply referencing or alluding to one or the other. Images, and the elements of which they are composed, use and rely on figured worlds just as we do when we speak or write. Figured worlds are simplified models or stories that we take for granted and that help explain how things are or should be in the world when they are ‘‘typical,’’ ‘‘acceptable,’’ or ‘‘normal’’ or, in some cases, ‘‘good’’ (we easily slip from ‘‘normal’’ and ‘‘acceptable’’ to ‘‘good’’). For example, there are a good many movies, especially made in the United States, that accept and play on the figured world that, if you just try hard enough and don’t give up (don’t give up your dream!), then in the end you will win, a common variant of what we called the ‘‘success model’’ in Section 4.9. The Karate Kid movies or the Rocky movies would be good examples. Finally, images, just like when we speak or write in language, are always part of Discourses, if the images are meaningful and communicative. Images are associated with words, settings, and other sorts of objects in the service of letting people enact or recognize different sorts of socially-significant identities and activities (practices). Just

as words need to be combined with other things (like ways of acting and interacting or using various sorts of objects or tools) to enact an identity, so, too, for images. A video gamer uses the images in a game to trigger certain actions, strategies, and values and, in the act, enacts his or her identity as a gamer in the Gamer Discourse. Someone else-say, a politician-sees only the images as content he wants to ban in his role as a moral crusader in a certain sort of political Discourse. So we always want to ask how images help people to enact different Discourses, how they seek to get people to recognize different Discourses, and how they seek to get people to participate in different Discourses. So far I have left out words. But many images contain words as well. When an image contains words, as in a typical ad or video game, the words play two roles. In one role they are elements in language that we can analyze along the lines of this book. In another role they are elements in the image and need to be analyzed as part of the image. We always want to ask what do the words add to the image (or its elements) and what does the image (or its elements) add to the words and how and what did combining words and image communicate that could not have been communicated (at least not in the same way) by images or words alone. After you have found situated meanings, ‘‘social languages’’ (styles), intertextual references, figured worlds, and Discourses at work (and play) in an image or multimodal text, then ask how these are carrying out our seven building tasks captured in our seven building tools. That is, how are situated meanings, social languages (styles), intertextuality, figured worlds, and Discourses being used to build significance (Tool 14), activities (Tool 15), identities (Tool 16), relationships (Tool 17), politics (Tool 18), connections (Tool 19), and sign systems and knowledge (Tool 21)? As an example, let’s consider ads, which are easy cases. It is clear that some ads seek to give significance to products beyond what people would otherwise give them. Ads are an activity, not just a communication: they are selling things. Ads often seek to get viewers to assume a certain identity, for example, as the sort of rich and powerful person who would drive a Hummer (see below). Ads are often designed to appeal to a certain customer niche or demography, often defined in terms of life styles (e.g., young urban professionals or middle-class Latinos) and, thus, seek to use and even help create a sense of people being related in certain ways to others in certain ways. Ads, of course, seek to connect things that, in reality, may have little connection. For example, some car ads seek to connect sex and cars when they place a ‘‘sexy'’ woman next to the car in the ad. Finally, ads often privilege certain styles of communication and ways of knowing the world. For example, the commercial I will analyze below uses humor (humor is a

distinctive communicational or sign system) to disarm viewers, and ads are certainly part of our ‘‘sound bite’’ culture that privileges short and pithy messages as a way of knowing over sustained argumentation and reflective thought. Now let me take a simple example to analyze along the lines I have suggested: a television commercial for a Hummer (a jeep-like vehicle based on the High Mobility Multipurpose Wheeled Vehicle-HMMWV or Humvee-used by the military; see https://video.google.com/videop lay?docid5-6547777336881961043# for the ad). This ad shows two monsters (a giant robot and a Godzilla-like monster) that look like they are from an old-fashioned black-and-white Japanese horror film. They are wreaking destruction on a city that looks like New York or some other skyscraper-filled city. Eventually the monsters meet up, fall in love, and have a baby. Their baby is a Hummer. The ad ends with the words (the only words in the ad): ‘‘It’s a Little Monster.’’ This ad was aired as a commercial during the 2006 Super Bowl. The two monsters and the city are clearly elements (though, of course, composed of other smaller elements). Within the ad, the monsters take on a number of situated meanings, one of which is surely ‘‘tough, hard, destructive beings that transcend the human laws and constraints of a city.’’ When they fall in love, we get a situated meaning like ‘‘they are more human-like and soft than we thought,’’ which comes as a surprise after the first situated meaning. The elements in the ad-the monsters and their destructive actions, the city, and the black-and-white film-like images-all create a pattern that resembles (perhaps in a ‘‘tongue in cheek’’ way) the style of an old-fashioned Japanese horror film. In fact, since there is no reference to Hummers at all until the end of the commercial, we viewers may be wondering why we are watching something that looks more like a grade B horror film than an ad for a car. This is the commercial’s style or ‘‘social language.’’ Of course, the fact that the ad has used the style of a Japanese horror film is also a clear intertextual reference to such films and to the popular culture of which they are a part. Later the two monsters act out a number of scenes that are a clear intertextual reference to a typical boy-girl romance story of the sort we have all seen in many movies. Since this sort of story is not what we expect in a horror film (at least from the monsters), this causes surprise based on what the earlier horror film intertextual reference has communicated to us. The romance part of the commercial relies on a common figured world about romance. A male-like being romances a female-like being. They both act like young people on a date and, indeed, act out stereotypical dating activities. The ad only needs quick shots here because so many viewers can fill in the whole story based on their figured world

for romance (a figured world that ironically has been partly informed by movies, though not by monsters in horror movies). It is impossible to understand this ad just on the basis of its images (and final words). Just as with language, we need to ask what Discourse or Discourses in society it is part of. What sorts of identities and activities (practices) is it trying to enact, get us to recognize, and asking us to participate in? How is it using not just its images and words to do this, but ‘‘dancing’’ (coordinating) with other things to accomplish this? This ad was shown during the 2006 Super Bowl (the U.S. professional football championship), one of the most expensive and most watched places to advertise in the world. Ads during the Super Bowl are expected to be entertainment and even ‘‘art’’ (at least, popular art), not just a sales pitch. Such ads get a lot of publicity and are an attempt to position the company and its products as having a ‘‘cool,’’ ‘‘modern,’’ ‘‘with it’’ (in terms of popular culture) identity in a high-tech, global world. Such ads are trying not just to be in such a Discourse, they are trying to create it and adapt (even co-opt) it to sell their products. Hummer-in this ad and in its other ads-is also associated with both a military Discourse (because Hummers came from and mimic military vehicles) and a Discourse of upper-middle-class wealth and power (because Hummers are very expensive and get very poor gas mileage; in addition to the fact they are much bigger than most cars, and thus Hummer drivers are more likely to survive in a crash with other drivers driving smaller cars). The ad we are analyzing has clear references to the military Discourse (identity) through its warfare/destruction images. After all, the baby Hummer is the child of two destructive forces and the implication is that it could help in that destruction, the way all children might follow in their parents’ footsteps. The Discourse of upper class wealth and power seems not to be referenced in the ad, other than through the association of a Hummer with such big and powerful objects as the monsters. However, the ad ends with the ‘‘daddy monster’’ (the robot) putting the Hummer down and watching it drive off in a way that looks to me just like a kid putting a toy car down, pushing it, and watching it move. This implies that Hummers are the ‘‘toys’’ or the ‘‘toy cars’’ of big and powerful ‘‘people.’’ Their ‘‘toys’’ are bigger than regular people’s cars, just as they themselves are bigger (richer, more powerful) than regular people driving ‘‘small cars.’’ To go further with Discourses we would have to study how people consume the Hummer ad to enact and recognize certain sorts of identities in the world. For example, I myself enjoy the ad, but as part of my values and actions in the world as someone who sees himself as part of a certain sort of environmental or ‘‘green Discourse.’’ I see

the commercial as both humorous in its own right, but also unintentionally (or is it really unintentional?) humorous in how it lets me read Hummers as destroyers of our environment, and the people who buy them as ‘‘monsters.’’ My response to the commercial allows me to enact this anti-Hummer Discourse, an irony given what General Motors (who owned Hummer in 2006, but no longer) spent on the ad. It is, of course, possible that the ad is ‘‘thumbing its nose’’ at people like me by saying something like ‘‘See, people who driver Hummers can destroy the environment and get away with it-people like you don’t matter and wouldn’t buy a Hummer anyway.’’ I have not dealt with the words in the ad: ‘‘It’s a Little Monster.’’ These words are an intertextual reference to a common ‘‘mean’’ thing parents say about their children but intend in an endearing way. This fits with the ‘‘human-like and soft romance’’ meanings and figured world in the second part of the ad. At the same time, it literally means the Hummer is, like its parents, a monster of the type that wreaks havoc on cities in old Japanese movies. This fits with the earlier part of the ad where we see the monsters as hard, powerful, and destructive. In the end, the images and words of the ad make us see a Hummer we own, or can own, as a child (or a ‘‘toy,’’ when we consider the last image in the commercial) with the potential for great power. It appears the ad’s ‘‘author’’ cares less that this power might be destructive than that it is mighty, ‘‘world shaking,’’ and impressive, which is what I think Hummer owners must believe (given the damage they do to the environment) about their cars and themselves. Even from what little I have said about the Hummer commercial, it is clear that situated meanings, social languages (style), intertextuality, figured worlds, and Discourses are tools that can be applied to images, as well as language. I have tried to hint how each of these theoretical tools is being used to engage in some of our building tasks, for example, how the situated meanings, social language (style), intertextuality, figured worlds, and Discourses recruited in the ad or by its ‘‘readers’’ are building things like significance, activities/practices, identities, relationships, politics, connections, sign systems, and ways of knowing. I do not have time here to take up each building task in turn. Indeed, it would be good practice for the reader to do so. So I encourage readers to apply many of our 27 tools to images and multimodal texts and not just to language alone.