ABSTRACT
Achieving safety in any socio-technical endeavour requires accurate data about the wayan organisation functions in normal operations and how it responds to threat and error. In aviation, a number of sources of operational data are available, but each provides a different and incomplete view of the organisation in action. Airlines have historically relied on data from their training departments (performance in training), assessment during scheduled flights (line checks), and formal proficiency checks conducted in simulators. These data provide accurate information on a pilot's technical competence and ability to respond to particular challenges. In Line Oriented Evaluation (LOE) the data demonstrate the ability of a full crew to respond to normal and abnormal situations during simulated flights. Data are also obtained from line checks in which an evaluator grades performance during normal flight operations. These sources provide valid information on pilots' ability to perform when being evaluated in a jeopardy situation (i.e., one where poor performance could result in disqualification). The limitation of data obtained in this fashion is that they do not inform organisations how crews behave when they are not under formal surveillance. In other words, are the practices taught and exhibited during formal evaluation routinely followed by crews not under scrutiny?
