ABSTRACT
The Challenge of Integrating Knowledge The issue of evaluation in public policy is a complex and deeply problematic area of both theory and practice. In this chapter we take a very broad view of evaluation to include policy analysis (as a field) as well as an aspect of policy planning. In simple terms evaluation is about ‘what works’, when, and for whom. Evaluation is the activity through which policy makers and those involved in policy-driven research endeavour to get information and knowledge out of problems so as to better understand the ways in which policies, programmes, and projects have brought about, or failed to bring about change, or how a proposed intervention is likely to impact on a given problem. How this task of accessing knowledge and using evidence may be approached, of course, depends on what kind of analytical frames are being used by policy makers and policy analysts.Evaluation is fundamentally a process of valuing, and different frames of value inevitably generate different ways of thinking about the problem of finding out what works, when, how, and for whom. Evaluation, therefore, directs our attention to the issue of (what one of the founding fathers of the policy sciences, Harold Lasswell, saw as) the task of integrating knowledge so as to contribute to the démocratisation of humankind (Lasswell, 1970). In Lasswell’s sense evaluation involves ‘who gets what knowledge, when and how?’ (Lasswell, 1958).The central question to ask about evaluation, therefore, is: whose values get to dominate? As Francis Terry notes: ‘a consideration of “what
works” in transport policy is at least partially determined by your philosophical starting point and the key questions are easier to answer at the tactical level than to do so in terms of total strategy’ (Terry, 2000, p.192). When we are discussing competing ways of obtaining knowledge about policy interventions in public problems, we are focusing on a major issue of democratic governance: the production and use of knowledge about social, economic, environmental and other problems. The task of integrating evaluation research for the purpose of improving policy making, however, must not to be seen as an exercise in blurring the edges, so much as making the edges clearer. Integrating knowledge is fundamentally to do with designing methods, processes and institutions which can best serve to clarify competing arguments, ‘stories', forecasts, discourses, and knowledge claims.The task of the policy sciences, Lasswell believed, was to facilitate the integration of knowledge about human problems: the great challenge for democracy was to ensure that different forms of knowledge, and sets of values could be used to develop more communicative and knowledgeable forms of governance. However, given the fact that there are deep and profound disagreements and differences of opinion and values, the idea of ‘integrating’ what we know about evaluation seems a contradiction in terms. To what extent, for example, is it possible or indeed desirable, to ‘integrate’ different and apparently incommensurate analytical frameworks? Does integration involve the idea of one ‘big’ ‘general theory’ of evaluation? Does evaluation actually require the development of some kind of Esperanto, or common language in order to become more useful for policy makers? Clearly the answer to this must be an emphatic no. Different approaches to evaluation embody, as it were, different ‘assumptive worlds’ (Young, 1977) or ‘frames’ (Schon and Rein, 1994). When Lasswell stressed the importance of integration it was not to promote a kind of intellectual unity or theory of everything, but quite the opposite: integration for Lasswell involved a process of clarification. The aim of the policy sciences was to promote a greater clarification of the values which knowledge in, for and of the policy process embodies. In seeking to integrate knowledge of evaluation in, for and of the policy process, therefore, the aim should likewise be to clarify different ways of thinking so as to understand what approaches work best in what contexts, when, how and for whom.This chapter argues that if we are to better integrate our knowledge of evaluation we should proceed by recognising that our knowledge is located in a variety of different and competing frames. This idea of ‘frames’ is
drawn from Donald Schön and Martin Rein’s work on ‘frame-reflective discourse’. Policy frames, they argue, are: ‘the taken-for-grantedassumptional structures of policy research’ (Schön and Rein, 1994).
