ABSTRACT
Five issues in experimental design are considered in this chapter.
Nested Factors and Natural Groups. In one typical nested design, treatments are given to subjects in natural groups, as with classroom groups in education or hospital wards in medical science. Correct analysis requires that the error term be calculated from differences between group means. The practice of calculating the error term from differences between subjects is not valid because subjects within groups are not generally independent.
Random Factors. With a random factor, the factor levels are chosen at random from some population, of stimulus materials, for example, or therapists, or classroom groups. This helps obtain generality across the random factor. Indeed, random factor Anova can provide a statistical generalization from the random sample to the population, although this benefit may have unacceptable cost. It may usually be preferable, therefore, to use standard fixed factor Anova.
Reducing Design Size. Multiple factors must sometimes be included in a design, but multifactor designs tend to be large and costly, even infeasible. Fractional replication can reduce design size by using only a well-chosen fraction of the conditions in the complete factorial design. Fractional design, extensively developed by statisticians, has been underutilized by psychologists. The rationale is shown for a simple case and illustrated with an empirical study of serial belief integration.
For single subject analysis, Latin squares have unrecognized potential for reducing design size while balancing multiple variables
For independent groups, however, Latin square designs have limited usefulness.
Unequal n . When a factorial design has unequal n in different cells, the simple formulas of Chapter 5 are not applicable. This once confused issue is no longer a big problem because the complicated calculations for unequal n are now done by computer. It is suggested, however, that primary use be made of two-mean comparisons with confidence intervals instead of overall Anova.
Quasi-Experimental Design. In quasi-experimental design, treatments are given to groups that differ preexperimentally. Treatment effects are thus confounded with uncontrolled/unknown differences between groups. Common methods to “control for” or “partial out” uncontrolled variables are seldom justified. Field science with nonrandomized groups has high importance, but it requires high expertise.
