Evaluating what works in distance learning Introduction 113; Distance learning 113; Evaluation approaches 115; The computing course 115; Evaluation 116; Lessons learnt 117; Examples of evaluation data 118; Use of online questionnaires 119; Conclusions 121; Acknowledgements 121;

Distance learning for any subject brings its own challenges in supplying learners with the information they need in a form they can use. The separation of the learner from the designers of the course means that extra effort should be applied to find out what is working well. The Open University (OU) has an approach to producing its own ‘classic’ course which is to plan for an eight-year life, invest in quality materials, carry out developmental testing or piloting of those materials and then monitor the students at the end of each year in the early stages of presenting the course. This has proved a robust process, but as the university and its students change, the model for the courses and how we evaluate them also has to change. In the computing department this was recognized and a new approach to courses has been advocated. Computing courses are designed in the expectation that they will evolve with changes occurring at each annual presentation. For those changes to be effective, the course team needs feedback at the right time; it is no longer sufficient to know what students think at the end of a course – rather, information is needed as the course is presented to look for problems and work on their solution.