ABSTRACT
This chapter discusses the foundational areas described in the Standards for Educational and Psychological Testing. It addresses the foundational area of reliability with an emphasis on the role and implications of reliability in the context of rater-mediated assessments. Spearman's research on reliability was motivated by a quest for identifying uniformities over replications of measurement procedures. The chapter highlights the idea that the underlying concern in much of the research on reliability can be productively viewed as a concern with the invariance of measures over a variety of circumstances and varying conditions used to locate a person on the continuum of interest. It shows that reliability/precision evidence for rater-mediated assessments should reflect a coherent measurement framework in which ratings are evaluated in terms of the requirements of a measurement model with useful properties. The chapter discusses the reliability concerns that are highlighted in the several clusters from the perspective of invariant measurement, particularly as they apply to rater-mediated assessments.
