ABSTRACT
This chapter provides an overview of the history of mixed reality (MR) and augmented reality (AR) in medicine, particularly in image-guided interventions (IGI). Beginning with early work using head-mounted displays (HMDs) in the 1990s, it traces the evolution of AR techniques used in the operating room (OR) to fuse various imaging modalities with the real-world view of the patient. The earliest application of AR in an intervention integrated an ultrasound (US) image of underlying structures within the patient, and this was followed by a project that fused preoperative imaging with operating microscope images in the OR. Many additional approaches have been employed, and these methods use semitransparent mirrors, HMDs, projection systems, and pseudoholographic integral videography (IV) displays to facilitate the combination of multiple imaging modalities with the patient in an intuitive manner. As new and unfamiliar display technologies create challenges related to human–computer interfaces and perceptual psychophysics, the chapter presents some of the issues that can inhibit the appropriate intuitive visualization of the real and virtual worlds in the medical context.
