Two-Sample Tests for Counts in Several Categories
Where there is a logical order to the categories the chi-square and Fisher’s exact tests ignore it, which is a waste of information. A good alternative is the Kolmogorov-Smirnov two-sample test, which takes account of the order of the categories and is also much less sensitive to small sample sizes or to big disparities in counts between categories. Websites sometimes warn that the Kolmogorov-Smirnov two-sample test should only be used on continuous data and not on counts in categories, but that is a misunderstanding. The test assumes that the thing that is being measured has an underlying continuous distribution; the fact that it is measured using counts in categories does not invalidate the test. Clast roundness, for example, is often measured using Power’s scale, which comprises six classes, but the roundness of stones is a continuum, from extremely angular to extremely well-rounded. Similarly, questionnaire responses may be recorded in classes labelled from strongly disagree to strongly agree, but we can assume that underlying this classification, there is a continuum of levels of agreement.