ABSTRACT

Educational accountability systems and causal inferences are inextricably linked. This is because all accountability systems provide both some measure of the current status of student academic achievement, and the extent to which this status is changing each year. At the same time, students within schools are constantly being exposed to what we might think of as “educational treatments.” Examples of such treatments might be the curricular choices or teaching strategies taken within classrooms, or administrative policies invoked to regulate student behavior. In a broader sense, most accountability systems appear to conceptualize teachers and schools themselves as educational treatments to be evaluated. Once stakeholders are given systematic information about student achievement, it is a natural tendency to speculate about the causal role that one or more of these educational treatments has played in producing achievement levels and changes in these levels over time. It is equally natural to speculate about how educational treatments may have managed to increase (or decrease) student achievement. The problem, of course, is that the act of speculating is subjective and open to debate. Even in the context of a controlled laboratory setting, making valid causal inferences is difficult, and clearly educational settings are far from controlled. From a policy standpoint we must recognize that to some extent, causal inferences about changes in student achievement will necessarily involve a mixture of speculation and science. The aim of this chapter is to help distinguish between the two, and to describe some approaches that help shift the causal inferences drawn from educational accountability systems toward a basis with greater reliance on science than on speculation.