ABSTRACT
This conclusion presents some closing thoughts on the key concepts discussed in the preceding chapters of this book. The book focuses on using Rasch Measurement Theory in combination with a lens model for human judgment to develop, evaluate, and maintain rater-mediated assessments. There are quite a few traditional methods for examining rating quality, including rater agreement indices, intraclass correlations, kappa coefficients, and generalizability coefficients. Each of these methods provides a piecemeal look at selected aspects of rating quality without being guided by a measurement theory that incorporates the variety of rater effects. The book summarizes the foundational areas identified in the Standards for Educational and Psychological Testing and their implications for rater-mediated assessments and explores how raters are using the rating scales. The statistics obtained from Classical Test Theory within the test-score tradition are person, cue, and rater dependent, while Rasch Measurement Theory offers the possibility of rater-invariant assessment systems.
