ABSTRACT
Subjective.Methods.of.Performance.Measurement........................................................................ 150 Purpose.of.Performance.Measures..................................................................................... 150 Special.Properties.of.Performance.Measures.in.Simulators............................................... 150 Defining.and.Assessing.Reliability..................................................................................... 151
Data.Requirements................................................................................................... 152 Qualitative.versus.Quantitative................................................................................ 152
A.Qualitative.Index............................................................................................................. 153 Quantitative.Indices............................................................................................................ 154
Is.the.Difference.in.Means.Meaningful?.................................................................. 154 Correlation.Coefficient............................................................................................. 154 Intraclass.Correlations.............................................................................................. 155 Number.of.Judges..................................................................................................... 157 Other.Designs.for.Assessing.Agreement.among.Judges.......................................... 158
Enhancing.Reliability.......................................................................................................... 159 Changing.the.Task.................................................................................................... 159 Rater.Training........................................................................................................... 159
Special.Problems.with.Simulators...................................................................................... 160 The.Gouge................................................................................................................ 160 Instructor.Attitudes................................................................................................... 160
Objective.Methods.of.Performance.Measurement......................................................................... 161 Automated.Data.Collection.Systems................................................................................... 162 Flight.Technical.Error......................................................................................................... 162
Deviation-Based.Metrics.......................................................................................... 162 Root.Mean.Square.Error.......................................................................................... 163 Number.of.Deviations.and.Time-Outside.Standard................................................. 163 Time.within.FAA.Practical.Test.Standard............................................................... 165
Non-FTE.Measures............................................................................................................. 166 Rates.of.Change........................................................................................................ 166 Control.Input............................................................................................................ 166
Summary........................................................................................................................................ 167 References....................................................................................................................................... 167
Performance.assessment.is.a.key.element.in.simulation..In.a.training.context,.performance.assessment.will.lay.the.foundation.for.feedback.to.the.pilot.or.flightcrew,.and.in.a.research.context,.it.is. typically.the.key.to.assessing.the.impact.of.various.factors.of.interest,.such.as.training.or.equipment. design..There.is.no.single.way.to.measure.performance,.and.practical.issues.typically.limit.the.type. and.amount.of.performance.data.that.can.be.collected.during.a.simulation.session.
