ABSTRACT
In this chapter we describe Pearson’s Continuous Flow (CF) system used for automated essay and short answer scoring, which uses the Intelligent Essay Assessor (IEA) engine for machine learning and scoring. The CF system provides a layer of automation on top of IEA and dynamically integrates human scoring with the IEA machine learning in ways not previously possible. As its name implies, it was designed to be a system in which data continuously flows between teams, systems, and organizations. It has been in operation at Pearson for several years and has resulted in reduced cost, lowered time-to-market, and increased flexibility in product offerings. We describe the drivers behind the development of the CF system, the design principles that were implemented, and the impacts it has had upon test administration, human scoring, and automated scoring systems in terms of systems design, workflow processes, and organizational considerations.
