ABSTRACT
Kress and van Leeuwen (2001: 2) observe that in digital documents ‘the different modes have technically become the same at some level of representation, and they can be operated by one multi-skilled person, using one interface, one mode of physical manipulation’. This points to a unique characteristic of the multimodality of digital text. It is a medium that can be used to encode multiple visual and sonic modes of expression and reproduce them in the form of integrated texts, using a single device that coordinates visual display with sound reproduction. This device may be a desktop or laptop computer, but it may also be a television set, a tablet, a mobile telephone, or a digital game console. In this sense, digital text refers not to the written text that that we see on, for example, a computer screen, but to various forms of coding and encoding analogue modes of expression, including programming and mark-up languages, and digital image, audio and video formats. Digital text encoding translates visual and sonic modes into strings of alphanumeric and graphic characters and, ultimately, into machine-readable binary code. Detailed discussion of digital text processing is beyond the scope of this book, except to note that this capacity to encode any visual or sonic mode underlies Kress’s observation that different modes ‘become the same’ in digital text. Because of its capacity to encode a range of semiotic modes and reproduce them as integrated text, digital text is the medium of multimodality par excellence . This capacity has emerged over time as a consequence of technological advances in data processing and in conjunction with the history of the multimodality of the web outlined at the beginning of Chapter 3 .
