Socrates funded project ‘Innovative Approaches in School Evaluation’ and this chapter draws on that work (see also Tiana et al. 1999; Smees & Thomas, 1999).
17.2 The Development of the Lancashire Value Added Project The original impetus for Lancashire LEA to begin to look at new ways of evaluating schools came from the external pressure of the UK conservative government’s education policy in the late 1980's to increase the public accountability of schools. League tables of secondary school performance in England and Wales, based on the national system of General Certificate of Secondary Examination (GCSE), were planned to be published nationally for the first time in 1992 by the Department for Education (DFE) and a few poorly achieving Lancashire schools had been told they were going to be highlighted in the press. As a result of this climate a collaboration between LEA advisers and a group of secondary head teachers was set up to discuss a fairer way of assessing school performance than the raw examination league tables. An additional pressure from central government came from a report by Her Majesty’s Inspectorate (HMI) which criticized the authority for not using data enough. One LEA adviser reported:
“it was a challenge to us to start to improve the way in which we used data” (LEA adviser and project manager)
The initial development work for the project involved a simple statistical approach, looking at the aggregated school means for attainment entry score at age 11 (Year 7) using the NFER Cognitive Abilities Test, compared to mean GCSE score at age 16 (Year 11) to examine the progress made by secondary pupils over a five year period. There were a number of methodological flaws with such an approach, and many schools soon began to question the appropriateness of a method that did not look also at the effect of pupil background factors such as social class and gender on attainment. It was at this stage that the LEA decided to seek help from external consultants and university academics. From this new collaboration the Lancashire Value Added Research Project was set up in 1992. The aim was to create a system that could contextualize raw GCSE results, taking into account both prior attainment and pupil background factors. After an initial pilot analysis in 1992 based on 11 secondary schools, the main Value Added Research Project was set up in 1993 involving 87 schools. Since 1994 all 98 Lancashire secondary schools have been involved in the project. 17.2.1 The rationale of the evaluation system: accountability versus improvement The LEA wanted primarily, a robust, ‘hard’ quantitative method of assessing schools’ attainment in a fairer context. They were particularly fortunate in that they already had in place an excellent pupil tracking system, whereby all pupils in the LEA mainstream schools took the NFER Cognitive Abilities Test at entry to
secondary school. It was also possible to collect a number of other pupil background details as well as the GCSE outcome results in order to carry out a value added analysis of the relative progress of pupils during their time at secondary school. As one of the LEA managers of the project points out:
“It’s very powerful feedback to a school on their performance for them to have to face up to the fact that they may not have been doing as well as they could have done and they would have to do something about it. For quite a number of school it still is, never mind was, a real eye-opener for them” (LEA adviser and project manager)
But the evaluation process was not intended for external accountability purposes, rather a tool for internal accountability and school improvement, in terms of assessing the performance of different subjects and groups of pupils as well as the whole school. The LEA tries extremely hard to encourage schools to use the Value Added data confidentially for ‘internal purposes only’, not to disclose such information to parents or the press, to prevent any of the negative aspects of the raw league tables:
“ A key element within it [the project] has been the integrity of the data because what we have never wanted to do is to publish an alternative league table” (LEA adviser and project manager)
17.2.2 Management of the project The management structure of the Value Added Project within the LEA is shown in Table 17.1. A key issue for the Value Added Project was to continue to include schools in the decision making process, and as a consequence the Value Added Working group was set up comprising of a group of secondary heads and the LEA project manager. The group meets two or three times a year to discuss new additions to the project or useful changes.