Module 11. Test Bias, Unfairness, and Equivalence
As we noted in Module 1, applied psychological testing is as much a political process as it is a psychometric one. Not surprisingly, then, accusations of test bias and unfairness surface on a predictable basis whenever a test is used to make an important decision affecting people’s lives. Some laypeople have used the terms test bias and test fairness interchangeably. However, a series of articles from the professional testing literature of the late 1960s and early 1970s clearly distinguish the two concepts. Test bias is a technical psychometric issue that focuses on statistical prediction, whereas test fairness is a sociopolitical issue that focuses on test outcomes. The concept of test bias has been operationalized in several ways (including differences in subgroup test means or validity coefficients); however, the consensus definition or current standard is what is known as the Cleary model (AERA/APA/NCME, 1999; Aguinis, Culpepper, & Pierce, 2010). Namely, one determines if a test has differential prediction for one group versus another by means of moderated multiple regression (MMR) analysis. Specifically, we are looking for possible subgroup differences in either regression slopes or Y intercepts. It is up to us, as evaluators of the test, to determine what subgroups are relevant. However, it is most common to examine so-called “protected” subgroups of test takers. These are typically demographically determined subgroups that receive protection by law. Thus, test bias is most commonly examined in subgroups formed on such factors as age, sex, or ethnicity.