ABSTRACT
This chapter focuses on objective performance assessments (OPAs), and suggests that OPAs are one of several data sources that should be employed when evaluating training effectiveness. Models of general training evaluation and effectiveness evaluations of flight simulator training specifically typically adopt a multi-faceted approach that uses several sources of data. OPAs possess several distinct strengths when compared to subjective measurements, such as absence of bias, greater precision, and automaticity of data collection. Perhaps the greatest strength of OPAs is the absence of personal bias and shifting frames of reference that pervade subjective ratings, which leads to low inter-rater and intra-rater reliability. Effective employment of OPAs requires an understanding of the measurement process. Because the number of variables that can be measured is virtually endless, it becomes paramount to identify and quantify those variables that are deemed to be most essential to training effectiveness and most directly measurable by an automated system.
