ABSTRACT
Pre-operational testing is the thoughtful, deliberate, and systematic process of collecting and analyzing evidence to support the validity of an assessment prior to that assessment’s operational use. Using quantitative and qualitative research methodologies, it is the process of conducting research to find answers to questions about, or to confirm the adequacy of, assessments, instruments, and tests currently under development. As such it shares much in common with all research endeavors, yet with a major and critical difference: its objective lens focuses ultimately on the test rather than the test takers and test score users, although all stakeholders may play a role in pre-operational testing. While there are many terms used to describe aspects of pre-operational testing, in this article we
will simplify the discussion by referring to two phases of the research. Pilot testing encompasses research on the test that is exploratory, conducted during a phase in which major or minor changes to the assessment and its accompanying pieces (e.g., scoring rubrics, test administration conditions, item and task formats) may still be made. Field testing refers to research on the test that is confirmatory, a final check that everything is working as intended and the test is ready to become operational within the context for which it is being developed. This latter definition by no means implies that research on the language test qua language test then ceases. Rather, it means that the test developer has made an evaluative judgment that sufficient evidence has been gathered to claim that the test can now be used for its intended purposes without further development. In other words, when pilot testing and field testing cease, the test developer implicitly claims that the instrument itself is now ready to be used to provide data which can lead score-users to making valid interpretations or taking appropriate actions about test-takers on the basis of their performances on the test. Together, pilot testing and field testing constitute pre-operational testing.
