ABSTRACT

In this chapter we characterize human rater scoring as an instance of human categorization processes related to category learning. We review key aspects of human categorization, including features of human natural categories and the cognitive and neurological systems that create them, and summarize this work via four core principles. We use this foundational work as a basis for understanding the processes by which both human raters learn to score constructed responses and the types of automated scoring techniques that have been developed to model the resulting ratings. We characterize three such approaches: (a) rule-based systems in which the scoring rationale is explicit, (b) statistically based machine learning techniques in which the scoring rationale is implicit, and (c) hybrid approaches that are combinations of the two. The framework laid out in this chapter can thus serve as one mechanism to understand meaningful distinctions in descriptions, enhancements, and applications of automated scoring techniques.