ABSTRACT
All interactions are multimodal. Imagine, for example, a simple two-person interaction, a conversation with a friend. During this interaction, you are aware of your friend’s spoken language, so that you hear the verbal choices, the content, the prosody, and the pitch. You are also aware of the way that your friend is standing or sitting, the way that your friend is nodding or leaning back or forward; you are aware of your friend’s facial expression, and clothing, just as you are aware of the environment in which this interaction takes place. If there is music playing in the background, even though you are not focusing on the music, you are aware of it. All of these elements play a part in this conversation. You may react to the words that your friend is speaking as much as you may react to your friend’s facial expression or the posture that your friend is taking up towards you. You may speak quickly or slowly, depending on the music playing in the background or the given environment that the interaction takes place in.
