ABSTRACT
Introduction ......................................................................................................... 118 Sample Design ..................................................................................................... 119
Population Definition .................................................................................... 120 Multistage Sampling ...................................................................................... 122 Sample Size Determination ........................................................................... 125 Stratification .................................................................................................... 126 Systematic Sampling ...................................................................................... 129 Probability-Proportional-to-Size Selection ................................................. 130 Classroom Samples ........................................................................................ 132 The Overall Sample Design .......................................................................... 133
Weighting ............................................................................................................. 134 The Purpose of Weighting ............................................................................. 134 Base Weights ................................................................................................... 134 Nonresponse Adjustments ............................................................................ 135
School Nonresponse Adjustments .......................................................... 136 Student Nonresponse Adjustments ........................................................ 137
Variance Estimation ............................................................................................ 143 Replication Variance Estimators in International LargeScale Assessments .......................................................................................... 144
The Jackknife .............................................................................................. 145 Balanced Repeated Replication ............................................................... 147
Some Practical Considerations in the Use of Replication Variance Estimators ........................................................................................ 148
Odd Number of Primary Units (Schools) .............................................. 148 Large Numbers of Primary Units ............................................................ 148
Software for Variance Estimation ................................................................. 149 Conclusion ........................................................................................................... 151 References ............................................................................................................. 152
Many educational assessment programs are conducted with the aim of obtaining results at the student, school, and administrative unit level. Such “high stakes” assessments may be used to make decisions about individual student progress through the education system, or as a tool used in the evaluation of teachers and schools. In such cases, generally every student in the population of interest is assessed, as results are desired for each student. In these circumstances there are no issues of sample design and selection involved, and no issues related to the need to provide analysis weights. In large-scale international assessments, however, almost invariably the goals of the study do not include the provision of individual student achievement results for all the individuals in the population. Rather, as is clearly demonstrated in the rest of this book, the purpose is to make inference about a whole population. This extends to interest in providing results for a wide variety of population subgroups, examining the distribution of achievement within and across these subgroups, and exploring the relationship of student and school characteristics to achievement.
