ABSTRACT
This study forms part of a substantial body of research into the access test, a test of English language proficiency for prospective migrants to Australia. The focus of this study is a trial version of an oral interaction subtest of access: The test is developed in two formats, live interview and tape based, which are used interchangeably.
The study examines whether both target and non-target-like subjects within the trial population can be used to provide evidence of test validity. First, the study examines the extent to which different groups of test-takers vary in their reactions to the test. Second, the question of whether specific test-taker characteristics affect performance is considered. Whereas the first question relates primarily to face validity (and the perceived comparability of the two formats), the second has implications for construct validity.
Questionnaires, seeking biographical details and reactions to specific aspects of the test, were administered to 94 trial candidates. All trial candidates had attempted both test formats.
Feedback indicated a clear preference for the live interview format. However, the positive reactions to the target-like section of the trial cohort indicates that the both formats have high face validity for this group. The results also suggest that the workplace orientation of the test is important for face validity.
210As expected the target-like group did not perform better than the non-target-like group. A large difference can be seen between the performance of Asian and European language speakers in the non-target-like group. However, language background was not a significant factor in the performance of the target-like group.
Test scores may mean different things for different people…. The notion that a test score reflects a single uniform construct interpretation … becomes illusory.
(Messick, 1988, p. 55)