ABSTRACT
Previous chapters dealt with the single-valued phenomena. However, multi-valuedness is quite common in characterizing texts, people, or situations. Any dictionary lists multiple meanings of words, people may have more than one hobby, publications may have several co-authors, and one situation may be described from different perspectives by its participants. Forcing observers to pick one of several equally valid accounts of phenomena is a well-known source of unreliability due not to observers’ incompetencies but the lack of an appropriate data language that fails to accommodate multi-valued phenomena.
Multi-valuedness needs to accommodate sets of values assigned to units of analysis. When comparing two sets of values, the values they share should not be counted toward disagreements. Calculating the disagreements in multi-valued reliability data requires a specialized difference function that embraces the metrics of the values and the sets in which they occur. This chapter develops this function, defines the appropriate observed and expected disagreements, and exemplifies the computation of a multi-valued α for the replicability. It goes on to develop multi-valued α-agreements for individual values in sets, for accuracy, and for sets of values regardless of their size.
