ABSTRACT
Actual developments in information and communication technologies (ICTs) are driven by two trends fostering fundamental paradigm changes in interaction and information processing. On the one hand, smartphones and mobile computing technologies offer highly sophisticated platforms integrating multimodal sensory and powerful central processing units (CPUs); on the other hand, information is structured, geo-referenced linking information to speci–c locations within our environment. Thus augmented reality (AR) has become a key technology as it analyzes the sensor data (camera, global positioning system [GPS], inertial) to derive the detailed pose of the mobile system, with the aim to correlate our real environment to the geo-referenced information space. Previous research has shown that AR is a powerful technology to support training because instructions on how to perform a speci–c procedure can be directly linked to the camera-captured environment. Various approaches exist in which the trainee is guided step by step through procedures, but these systems act more as guiding systems than as training systems and focus only on the trainee’s sensorimotor capabilities. All these facts lead to the need for ef–cient training systems that accelerate the learning and acquisition of the skill process. Furthermore, these systems should improve the adjustment of the training process to new training scenarios and enable the reuse of existing training material that has proven its worth. In this chapter a novel concept and platform for multimodal
Introduction .............................................................................................................. 81 Capturing Technologies ........................................................................................... 83 Capturing of the Camera Pose .................................................................................85 Capturing of Activities .............................................................................................85 Augmented Reality and Teleconsultation ................................................................87 Augmented Reality-Based Training .........................................................................87 Summary and Conclusions ......................................................................................88 References ................................................................................................................ 89
AR-based training is presented, that fuses capturing and rendering technologies in the following way (cf., Figure 6.1):
• Capturing: The camera is used to capture activities (of an expert) within a large-scaled environment. Video cameras are used to track tools or to register motion, and applied forces and torques are registered. The captured trajectories of tools can be represented within a three-dimensional (3D) animation, or the captured information is linked to the real environment using the Virtual Post-It metaphor. Virtual Post-Its link contextual information to speci–c parts of the real world. Starting with forces/torques to be applied and resulting in multimedia illustrations including audio/video –les, the Virtual Post-Its can document complex workŸows.
