ABSTRACT
The methodology typically called standard setting consists of two parts: (a) a structured means for collecting judgments from individuals about the desired level or levels of performance on a test and (b) a statistical process for converting the judgments into a point on the reporting score scale corresponding to each level. The judgments are expected to correspond to a policy statement or descriptions of desired levels of competency. The judgmental tasks that provide the information for determining the point or points on the score scale are challenging, and participants in the process may not have much training or experience with these tasks. As a result, a substantial component of the time dedicated to a standard setting process is dedicated to training the participants to perform the required tasks. An important part of training is feedback that is given to help participants understand the tasks they are asked to perform and to provide information about how well they are performing the required tasks. The purpose of this chapter is to describe the kinds of feedback that are given during a standard setting process and the specific role for each kind of feedback. The impact of feedback on the process will also be discussed followed by specific recommendations about how feedback should be implemented within a standard setting process. Before dealing with the specifics of feedback, a brief overview is given about philosophical approaches to standard setting to set the context for giving feedback during the process.
