ABSTRACT
The objective of impact evaluation is to establish causality between an intervention and observed changes or effects, whether the changes are positive or negative, intended or unintended and direct or indirect. To accomplish this, impact evaluation must at a minimum undertake three tasks: first, determine the extent of change that has taken place since the intervention started; second, establish a causal link between programme or project activities and the observed changes; and, third, discount rival hypotheses or alternative feasible explanations that could account for the observed change. When evaluating impact in complex systems (e.g. a large marine ecosystem), these three tasks pose a set of challenges that stem from phenomena such as unpredictable time lags between project end and evidence of impact occurring; the effects of factors interacting at different scales; and the actions of multiple actors. The combined effect of these challenges makes the extent of change difficult to assess and to attribute to the intervention being evaluated. This chapter illustrates how the Global Environment Facility (GEF)’s “Impact Evaluation of GEF Support to International Waters in the South China Sea and Adjacent Areas” employed a mix of conventional evaluation methods and complex systems analysis tools and concepts to deal with these challenges. The chapter also discusses the importance of assessing the progress made towards achieving impact by focusing on likely long-term effects and indirect impacts as the more relevant outcome of the GEF’s catalytic role, given the complex nature of the ecosystems and interventions that the GEF typically supports. The authors revisit key conclusions made when the evaluation was originally conducted almost a decade ago, examining which assumptions made then remain valid now. With the benefit of hindsight, the authors also reflect on the approach used to evaluate interventions in complex systems, which by their very nature will never be completely understandable or predictable.
