ABSTRACT

With expediencies such as rapid turnaround time and economic benefits associated with reduced scoring costs, automated scoring is widely viewed as on a path to replace human labor, like so many prior innovative technologies. As a result, the focus of automated scoring development has tended to be on validity or measures of accuracy as compared to human efforts. This chapter is focused on a somewhat nearer-term process, the transition from human to machine scoring and how the two scoring methods can be used concurrently. Previously, the authors have referred to the use of both human and machine scoring for a single assessment program as a “blended” scoring model (Lottridge, Mitzel, & Chou, 2009). The central idea is to see if two imperfect scoring methods can be mutually leveraged in some optimal way to improve the overall accuracy of scoring. Although the authors have not yet achieved that goal, the studies presented here can be taken as a progress report toward that objective.