ABSTRACT
This chapter has two themes: reducing error variability and order effects. The main source of variability is usually individual differences, which can be handled in four ways: good procedure; screening out extreme scores; blocking on some individual difference variable; and using repeated measures. Repeated measures faces problems of order effects, but in many cases order effects can be handled with Latin square versions of standard repeated measures designs.
Screening Subjects and Writing Instructions. Extreme scores are a major headache. In empirical perspective, extreme scores may result from shortcomings in procedure, from real individual differences in the population, or from outliers that do not belong. In statistical perspective, extreme scores run up the error variance, widen confidence intervals, and decrease power.
The first line of protection against extreme scores lies in experimental procedures to prevent them. Most important, of course, are procedures to establish task-subject congruence. Also useful are procedures to screen out extreme subjects or scores.
Block Design. If subjects are stratified, or blocked, on some individual difference variable, these individual differences can be factored out, thereby reducing the error term. Although somewhat limited in range of applicability, blocking has more potential than has been realized, especially in the limiting case of analysis of covariance.
Latin Squares. Latin squares are useful extensions of repeated measures design, in which treatment position is included as a design factor. Latin squares balanced for previous treatment go further to provide some analysis of carryover effects, that is, treatment—treatment interactions. This class of designs has unrealized potential for experimental analysis.
Within Subjects Versus Between Subjects Design. Within subjects design is highly desirable, for substantive and statistical reasons both. But within subjects design can suffer from transfer or carryover from one treatment to the next. As a consequence, within subjects design sometimes yields very different results from between subjects design, in which carryover effects cannot occur. Decisions about which design to use depend mainly on extra-statistical knowledge about the situation at hand. A number of empirical examples are given to help focus attention on this basic design problem.
