ABSTRACT
This paper describes a technique for developing and analyzing detailed models of complex student problem solving, and methods to measure the reliability and validity of these models. Specifically, we use the Interactive Multi-media Exercises (IMMEX) system to record the specific steps students use to solve open-ended problems. While IMMEX has been used in numerous academic disciplines, the research documented in this paper relies on biology and chemistry students. We analyze thousands of such performances using artificial neural networks as a data-clustering tool that aggregates student performances without a priori knowledge of those performances and without the limitations imposed by comparing these performances to “experts.” The resulting clusters serve as a rich source of assessment information, and can provide students and educators with the meaningful practical feedback necessary to improve learning. Finally, we analyze these clusters and explore the data features that influence the reliability and usefulness of such a tool.
