ABSTRACT
This chapter provides an initial look at the quality of instructor evaluations by estimating the reliability of both technical and crew resource management (CRM) evaluations given by instructor pilots to Navy pilots flying a full-motion simulator. A scenario was developed for CRM assessment in which specific, realistic events were embedded in the scenario for assessing the skills identified in the needs analysis. A flight instructor attached to the community who was heavily involved in training CRM was the primary architect for the scenario. The correlation between judges for each item was computed for each item type; that is, correlations were computed between judges over crews for the specific behavior items, the global events items, and the CRM items. Scales were formed for the mean judges' response to specific behaviors, the events evaluations, and each of the seven CRM dimensions.
