ABSTRACT
Ranking is a very popular way of collecting data. For instance, members of a panel are asked to rank eligible baseball players for the Hall of Fame. Typically, the best one is given rank 1, which is worth as many points as the panel may decide, and the worst one the last rank. Unlike paired comparisons, ranking becomes increasingly more difficult as the number of stimuli or objects to be ranked increases. For example, imagine asking a group of people to rank 20 flower arrangements in order of excellence at a garden show. Wouldn't that be a difficult task? Can you rank 30 politicians in terms of the confidence you have in them? You may be certain about your ranking of the first 5 or so, but you would not be able to swear to the accuracy of your ranking of, for example, the last 10. In contrast, paired comparison is easy no matter how many stimuli (objects) are involved, since the task is always to choose one out of the two in a pair. But as the number of stimuli, n, increases, the number of pairs, n(n−1)/2, may soon become hopelessly large. For instance, the total number of pairs for n of 10, 20, and 30 are respectively 45, 190, and 435, suggesting that some subjects may get tired of judging pairs of stimuli even when n is only 20. Therefore, it looks as though both ranking and paired comparison are limited to a small number of stimuli, say up to 15.
