ABSTRACT
Do automated essay scoring (AES) systems produce valid estimates of writing skill? How can researchers establish the validity of AES systems; what kind of evidence should be considered? Given the nontraditional nature of AES, it is tempting to think that such new methods require new forms of validity evidence. It is argued that traditional methods of demonstrating validity will work equally well in demonstrating the validity of AES. This chapter reviews the types of validity evidence that should be relevant for AES; reviews the existing validity evidence for specific AES systems; and discusses the types of additional studies that need to be conducted to demonstrate the validity of AES programs.
