ABSTRACT
The concept of UNDERSTANDardization was introduced by Sireci (2020b) to promote flexibility in educational test design, development, administration, and scoring; and to achieve the most valid measurement of the proficiency of each examinee. A key component underlying UNDERSTANDardization is research. Testing agencies must conduct sufficient research on the population to be tested to understand the different layers of heterogeneity that exist within the population (e.g., culture, educational experiences, languages, and linguistic styles), and incorporate this understanding into test design, development, administration, and scoring. In this chapter, we first describe the history, concept, and principles of UNDERSTANDardization. Next, we propose six research questions to help testing agencies gather the information they need to ensure the testing process is sufficiently flexible to allow all examinees to best demonstrate their proficiency. We then describe the various data sources that could be used to best understand the different types of examinees within the population to be tested as well as methods that can be used to gather and analyze these data. By conducting research to understand the entirety of the population of people to be tested, testing agencies can provide more valid interpretations of their proficiencies, which will lead to more fair and efficient testing practices.
