ABSTRACT
It must be clearly understood whether or not the audit report requires some quantitative measure. There are both advantages and disadvantages to the use of quantitative performance measures. On the positive side, that great scientific pioneer Lord Kelvin reminded us that we do not know anything about a subject until we can measure it. Clearly, it is advantageous when comparing two successive audits to see whether there has been some overall measurable change either upwards or downwards, particularly in large SHE management audits where there may be gains and losses in different aspects of safety, health, and the environment and it becomes very difficult to identify the overall progress without some measurable assessment. Quantitative results have been very effective in assessing performance against some stretch targets. It is noticeable that most commercially available SHE auditing systems involve a quantitative measurement system and some have the great benefit of allowing benchmarking against other similar organizations. Often these auditing systems are computer based. To their credit, most of the commercial systems also provide some qualitative feedback process. Unfortunately, through no fault of the supplier, their customers tend, in my experience, to concentrate on the use of the quantitative system. It is not clear to me why this is so; it could be because of the inexperience of the user or it could be that they are primarily used by managers who may wish to confirm their own preconceptions of how successful they are or even how unsuccessful their predecessor was. The problem with any quantitative system is that it is open to some level of interpretation. The arithmetic is robust, but the individual performance assessments will vary and can vary quite widely. The difference arises from the interpretations given by different assessors or auditors. Provided the same assessor is used, then the relativity between different audit scores will be consistent, but once the common factor of one auditor is removed, then meaningful comparisons become very difficult, unless some form of consistency checking is built into the process. The second concern about the use of quantitative measures regards their perceived accuracy. An audit result of 78 percent compliance compared with the previous audit result of 80 percent is often perceived negatively by the senior manager as an indication of deteriorating performance. This comparison needs to be put in the context of the accuracy of the audit scoring system, which is often no better than plus or minus 5 percent. With this interpretation, the audit result is seen as the same level of compliance as before. At least one leading SHE audit system overcomes this
problem by not declaring the actual numerical score, but by indicating a compliance level, with a group of numerical scores grouped into each level. This has the great benefit of removing the sensitivity to spuriously accurate small percentage changes and identifies only changes that are sufficiently significant to cause a change in scoring/compliance level. When providing audit “scores,” the auditor should avoid the temptation to declare the score at the exit meeting, even if you know it. The reason for this is that it will precondition the listeners. If the score is less than expected, the listener may become disgruntled, and if it is better than expected, the listener might conclude that there is not much to do. In either case the consequence can be the same-the listener stops paying attention to the other important messages at the exit meeting presentation. However, do make it clear at the entry meeting that you will not be drawn into a premature statement about compliance scores.
