ABSTRACT
What can the performance of a person in a given task, evaluated by a given rater, tell us about other scores this person might obtain on tasks that are similar in some ways but different in others, or scored by different raters? What might it tell us about other people, other tasks, or other raters, each similar in some ways but different in others? Generalizability theory (g-theory) addresses such questions and many other issues in assessment design and inference. This chapter discusses g-theory from two perspectives: (1) Sociocognitive psychology connects g-theory with the hierarchies of stability and variation across situations and persons that g-theory addresses. (2) From a Bayesian perspective, a g-theory model is an exchangeability structure for situated model-based reasoning. Special attention is given to situations with human raters, including extensions to IRT models with rater effects and illustrations from the Advanced Placement Studio Art portfolio assessment.
