ABSTRACT
Planning, as coordinator, integrator and mediator of space, has continued to seek a knowledge-driven and evidence-based approach to assess the achievement of policy objectives and intervention. The pertinence of indicators in programme monitoring and evaluation was made explicit by the European Commission (2000) when launching its New Programming period in 2000. Performance measures have also been widely used in North America (Hoernig and Seasons, 2004; Swain and Hollar, 2003). In Britain, central government has emphasised the importance of having robust indicators as part of performance management in the public sector (Audit Commission, 2000), including the latest spatial planning reforms (HM Government, 2004; ODPM, 2004, 2005a, 2005b). Presented as an elegantly precise set of numbers, the systematic and uniform
nature of indicators serves well as a policy instrument to justify resource distribution and to track progress made. As with other quantitative analytical methods, indicators tend to be perceived as part of the empiricist or positivist tradition. However, Wong (2006) argues that the exact nature of indicators is rather more complex and it is the unique blend of technical and normative rationality that makes indicators a “Jekyll and Hyde” character. This unique quality is particularly attractive to policy-makers because the concepts to be measured can be shifted, and the indicators used can be adjusted. Indicators can be defined as the operational definitions of some abstract
concepts (Carlisle, 1972) and they offer a guide showing how a particular issue is structured or is changing (Miles, 1985). This theoretical view serves the empirical school of thought well, as the emphasis is on finding a way to provide measurement of concepts and problems. Bauer’s (1966: 1)
emphasis on indicators as a yardstick to measure progress and goal achievement, however, adds a normative dimension to the empirical formula. The emphasis on value and goal-setting – the presupposition of certain innate knowledge – to benchmark against the measured result shifts the epistemological basis closer to that of the rationalists. The theoretical expression of these two definitions shows that indicators do not sit comfortably in either the empirical or the rationalist tradition in their pure form. Following the definition of the US Department of Health, Education and Welfare (1969: 97), indicators are subject to another epistemological turn as they are seen as “statistic[s] of direct normative interest … and [are] subject to interpretation”. This suggests that value judgement would be involved in viewing some effects as better or worse. More importantly, the norm of assessment is susceptible to change and interpretation. Following neo-Kantianism, this definition opens the argument of relativism, and stresses the importance of inter-subjective communication and interpretation of meaning. The value-laden aspect of indicators clashes with both the rationalist and
the empirical ideology, as the foundation of securing objective knowledge from belief, opinion and even prejudice is somewhat less convincing. This signals the underlying tension of indicators as policy instruments that are subject to the politicisation of interpretation and the possibility of manipulation even at the measurement stage through the choice of indicators, data sources and methods. Furthermore, it is important to note that indicators tend to be very good at picking up issues that can be best monitored through numbers (Wong et al., 2006a), but less effective to ascertain outcomes that are less tangible. Indicators are also best used as traffic lights to send out signals when policies are not delivered and may need to be reviewed to take into account changing situations (Innes and Booher, 2000). This then poses an interesting question – in what ways can indicators be
used to transform the quality of public discourse and inform policy decisions? This chapter, therefore, attempts to address the thorny question of whether we can develop a robust and reliable approach to evaluate the performance of spatial planning with a set of indicators. The central premise is to shift away from the mechanistic, single-loop policy evaluation model to a more analytical and collaborative framework that allows key stakeholders to express their vision in the policy formulation process as well as providing a feedback loop to frame policy problems. The discussion here is based on the findings of a recent research project2 (Wong et al., 2008) funded by the Department for Communities and Local Government and the Royal Town Planning Institute (RTPI). The chapter consists of five further sections. The second section focuses
on analysing the nature of planning policy problems, both theoretically and practically, via the case of England. The third part turns to examine the nature and purpose of different types of indicator in policy evaluation. The fourth section introduces the identified indicators for measuring spatial
planning outcomes in England. The discussion then turns to explain how indicators can be used to develop an integrative, double-loop learning evaluative framework to measure the effectiveness and coordination of spatial planning policy. Finally, a set of guidance principles are outlined to guide the development of future planning evaluation indicator frameworks.
