ABSTRACT

After a conceptual measurement design is determined, data can be gathered through the chosen design. These data can then be analyzed statistically to estimate the magnitudes of various sources of measurement error. The statistical procedure used in generalizability theory is the Analysis of Variance (ANOVA) procedure. Specifically, after data have been gathered, they are analyzed through ANOVA in the G-study stage of the analysis. The purpose of this ANOVA is to estimate variances due to the object of measurement, different facets of measurement, and the interactions among the object of measurement and facets. In the common application of ANOVA in inferential statistical significance testing, the purpose of the ANOVA process is to estimate F-ratios. Based on these F-ratios, decisions are made as to whether to retain or reject null hypotheses. In the generalizability application of ANOVA, however, F-ratios are of no interest at all. Instead, the mean square (MS) estimates are used as the bases to estimate the underlying variance components.