ABSTRACT
Virtual environments (VE) represent advanced, immerssive, human-computer interaction systems. Such interaction occurs through communication over several sensorial channels. Since the communication pathway between users and the simulation system groups several distinct channels, it is termed multimodal. The modalities used are primarily the visual, auditory, and haptic ones. The number, quality, and interaction between such modalities are key to the realism of the simulation and eventually to its usefulness. Thus the need for increased immersion and interaction motivates system designers to explore the integration of additional modalities in VE systems and to take advantage of cross-modal effects. The downside, of course, is increased system complexity, cost, and possible integration/synchronization problems. In addition, cross-modal interaction can lead to perceptual illusions that must be considered by system developers (see chap. 22, this volume).
