ABSTRACT
This chapter concerns α’s statistical behavior. It reviews α’s range, the part that is interpretable in terms of the reliability of data, and its scale. It then addresses the importance of variance, not only for data to be analyzable and answer given research questions but also to be less vulnerable to errors. It demonstrates how moderately compromised reliability data affect the simplest analytical results: frequency counts.
While it is easy to state the conditions under which α is statistically significant, α’s interpretability as a reliability coefficient also depends on satisfying at least four separate sampling issues. The units in the reliability data have to be representative of the population of data whose reliability is under consideration. Reliability data have to include an adequate number of replications. The qualifications of coders must be shared widely for data to be replicable elsewhere. And its coding instrument has to provide the information needed to answer given research questions.
The chapter then turns to two measures that contextualize the measured α in its probability distribution: Its confidence limit at a chosen level of statistical significance and its probability of their failing to be above a chosen minimum. The chapter offers two approximations to these measures, for nominal data by reference to the normal distribution and for all kinds of data by an algorithm for bootstrapping a distribution.
