ABSTRACT
The combination of visual and auditory information at the human-computer interface is a powerful tool for interaction. In everyday life, both senses combine to give complementary information about theworld. Our visual system gives us detailed information about a small area of focus, whereas our auditory system provides general information from all around, alerting us to things outside our peripheral vision. The combination of these two senses gives much of the information we need about our everyday environment. Blattner and Dannenberg (1992) discussed some of the advantages of using this approach in multimedia/multimodal computer systems: “In our interaction with the world around us, we use many senses. Through each sense we interpret the external world using representations and organizations to accommodate that use. The senses enhance each other in various ways, adding synergies or further informational dimensions.” They went on to say:
These advantages can be brought to the multimodal humancomputer interface by the addition of nonspeech auditory output to standard graphical displays (see chapter 14 for more on multimodal interaction). While directing our visual attention to one task, such as editing a document, using sound we can still monitor the state of other tasks on our machine. Currently, almost all information presented by computers uses the visual sense. This means information can be missed because of visual overload or because the user is not looking in the right place at the right time. A multimodal interface that integrated information output to both senses could capitalize on the interdependence between them and present information in the most efficient way possible.
