ABSTRACT
This chapter focuses on the “weighing” part—addressing the question of how much evidence is enough to support confidence in intended test score meaning and use. The wisdom of G. V. Glass et al.’s observation applies equally to consideration of how much evidence is sufficient to claim confidence in the intended meaning of test scores or that an intended test use is justified. Success in uncovering disconfirming evidence can reduce confidence in the intended score interpretation and may dictate reconsideration of the proposed interpretation—or substantial revisions to the test development process. A related validity concern that should be taken into account when evaluating the strength of the case for justifying an intended test use is a comparison of the validity evidence for the alternative measures that could be used for the same testing purpose. When alternatives exist, the extent of validity evidence supporting each of the alternatives provides one criterion for evaluating which, of the alternatives is more justifiably used.
