ABSTRACT

This chapter presents a number of alternatives to Cohen’s kappa, together with supplementary information which has to be reported with Cohen’s kappa: bias and prevalence. It deals with agreement and ordinal scales. The chapter discusses the analysis of categorical data with an underlying continuum and also presents a measuring instrument: the tetrachoric and polychoric correlation. It explains a general coefficient, Krippendorff’s alpha, that can be used in case of nominal scales but also for ordinal and continuous scales and also discusses a small number of relatively frequently used indices of agreement. The basic difference between the indices is the way the expected proportion of agreement is expressed. Kappa indices can also be used for ordinal scales, for instance, when listeners are asked to rate the severity of stuttering from ‘no stuttering’ to ‘mild stuttering’ to ‘severe stuttering’. Kappa and kappa-like indices (apart from Krippendorff’s alpha) are the coefficients used to assess agreement between these kinds of data.