ABSTRACT
Setting standards on tests or assessments is a social judgment process (Higgins, 1996) that involves decision making under uncertainty (Tversky & Kahneman, 1974). One or more panels of judges are assembled for the purpose of recommending what examinees should know and be able to do to achieve some valued end and to specify an associated score on a test that is regarded as an indicator of requisite knowledge and ability The panel that sets this test score often uses an indirect procedure that requires its members to make estimates of the statistical properties of test items (e.g. Angoff, 1971; Ebel, 1972; Nedelsky, 1954) or to make classification decisions about examinees (Livingston & Zieky, 1982), typically under conditions
in which they have incomplete information about the consequences of their decisions.
