ABSTRACT
It is difficult to put forth one 'best' recommendation for multiple comparison of means. It is important to make the distinction between comparisons that are planned before conducting the experiment and those that appear interesting after examining the data. Comparisons continue with progressively smaller subsets, down to sets of two means, until no more significant differences are found. A key feature of compromise involves the tradeoff between experiment-wise and comparison-wise error rates. The Scheffe method controls the experiment-wise error rate for all possible comparisons. It is very handy for conducting a large number of unplanned comparisons, or 'data snooping'. There are numerous tradeoffs between methods, depending on the type and number of contrasts to be considered. Any multiple comparison method only provides a method of assigning significance to the ordering of means. The difficulties arise with uncertain separation of means, and the scientist must understand the limitations of taking any particular test too seriously.
