ABSTRACT

Applications that wish to utilize the Kinect and the data streams that it produces can do so with the Kinect for Windows software development kit (SDK) [Microsoft 12]. To properly obtain and interpret these data streams, a developer must understand the mechanics of how the device operates and also have a clear understanding of what can and can’t be done with it. This article seeks to provide the theoretical underpinnings needed to use the visual and skeletal data streams of the Kinect, and it also provides practical methods for processing and using this data with the Direct3D 11 API. In addition, we will explore how this data can be used in real-time rendering scenarios to provide novel interaction systems.

2.2 Meet the Kinect We will begin our investigation of the Kinect by examining how its camera systems function. The camera system is essentially composed of a color (RGB) camera, an infrared (IR) projector, and an IR camera. These three elements are shown in Figure 2.1. These sensors, when used together, provide a form of a vision