ABSTRACT
In this chapter, we provide an overview of different methods used in scaling and norming essay scores starting with a general comparison of holistic and analytic rubrics. Next, scales applied for rating the quality of writing samples are reviewed with a focus on standardized writing development scales established for application in automated essay evaluation (AEE) providing for an examinee’s writing ability to be meaningfully tracked over time and across essay prompts. Methods for formation of scores in AEE are then overviewed. Common standard-setting methods are summarized and, finally, differential item functioning methods are discussed. Throughout the chapter, we raise validity issues that continue to exist today, and make recommendations for future work in the effort of producing meaningful scores in the process of scaling and norming essay scores in general and in AES.
