ABSTRACT
In this chapter we describe an exploration of the problem of automated scoring of process data from a game or simulation. Unlike a traditional test administration, this assumes that the test take is working on a remote computer which is streaming event log data to the scoring server. There are two complications that arise in processing streaming event data. First, because of networking issues, data may not all arrive at the scoring server or may be out of order or delayed. Second, the event log data are usually voluminous, and need to be summarized as a collection of observables before they are entered into psychometric models for scoring (e.g., Bayes nets, item response theory, or diagnostic classification models). The prototype software used to solve these problems for the educational game Physics Playground and the lessons learned from that application are discussed.
