ABSTRACT
This chapter introduces classification and regression trees (CART) to investigate the factors that determine listening test item difficulty. CART has been used extensively in education fields and machine learning but has received relatively less attention in language assessment. We discuss the major components of a reliable CART study such as articulation of theoretical frameworks, using cross-validation, estimation of fit statistics, and accuracy of classification. The chapter employs the construction-integration (CI) model of comprehension to measure item difficulty in a large pool of listening test items. The data comprised seven Michigan English Test (MET) listening tests comprising 321 items answered by 5039 international language learners. CART modeling generated 41 IF-THEN rules, which revealed nonlinear relationships between item difficulty and 12 independent variables (IVs) measured by Coh-Metrix. CART enabled us to show that the relationship between item difficulty and the 12 IVs is not linear, and different sets of rules would apply to predict the difficulty of different groups of listening test items.
