Advancing Natural Language Processing in Educational Assessment examines the use of natural language technology in educational testing, measurement, and assessment. Recent developments in natural language processing (NLP) have enabled large-scale educational applications, though scholars and professionals may lack a shared understanding of the strengths and limitations of NLP in assessment as well as the challenges that testing organizations face in implementation. This first-of-its-kind book provides evidence-based practices for the use of NLP-based approaches to automated text and speech scoring, language proficiency assessment, technology-assisted item generation, gamification, learner feedback, and beyond. Spanning historical context, validity and fairness issues, emerging technologies, and implications for feedback and personalization, these chapters represent the most robust treatment yet about NLP for education measurement researchers, psychometricians, testing professionals, and policymakers.

The Open Access version of this book, available at www.taylorfrancis.com, has been made available under a Creative Commons Attribution-NonCommercial-No Derivatives 4.0 license.

part I|73 pages

Automated Scoring

chapter 1|12 pages

The Role of Robust Software in Automated Scoring

ByNitin Madnani, Aoife Cahill, Anastassia Loukina
Size: 1.08 MB

chapter 2|16 pages

Psychometric Considerations When Using Deep Learning for Automated Scoring

BySusan Lottridge, Chris Ormerod, Amir Jafari
Size: 1.26 MB

chapter 3|27 pages

Speech Analysis in Assessment

ByJared C. Bernstein, Jian Cheng
Size: 2.88 MB

chapter 4|16 pages

Assessment of Clinical Skills

A Case Study in Constructing an NLP-Based Scoring System for Patient Notes
ByPolina Harik, Janet Mee, Christopher Runyon, Brian E. Clauser
Size: 1.27 MB

part II|49 pages

Item Development

chapter 5|13 pages

Automatic Generation of Multiple-Choice Test Items from Paragraphs Using Deep Neural Networks

ByRuslan Mitkov, Le An Ha, Halyna Maslak, Tharindu Ranasinghe, Vilelmini Sosoni
Size: 0.97 MB

chapter 6|17 pages

Training Optimus Prime, M.D.

A Case Study of Automated Item Generation Using Artificial Intelligence – From Fine-Tuned GPT2 to GPT3 and Beyond
ByMatthias von Davier
Size: 1.83 MB

chapter 7|17 pages

Computational Psychometrics for Digital-First Assessments

A Blend of ML and Psychometrics for Item Generation and Scoring
ByGeoff LaFlair, Kevin Yancey, Burr Settles, Alina A von Davier
Size: 1.91 MB

part III|40 pages

Validity and Fairness

chapter 8|15 pages

Validity, Fairness, and Technology-Based Assessment

BySuzanne Lane
Size: 0.94 MB

chapter 9|23 pages

Evaluating Fairness of Automated Scoring in Educational Measurement

ByMatthew S. Johnson, Daniel F. McCaffrey
Size: 2.55 MB

part IV|70 pages

Emerging Technologies

chapter 10|16 pages

Extracting Linguistic Signal From Item Text and Its Application to Modeling Item Characteristics

ByVictoria Yaneva, Peter Baldwin, Le An Ha, Christopher Runyon
Size: 1.20 MB

chapter 11|17 pages

Stealth Literacy Assessment

Leveraging Games and NLP in iSTART
ByYing Fang, Laura K. Allen, Rod D. Roscoe, Danielle S. McNamara
Size: 1.68 MB

chapter 12|17 pages

Measuring Scientific Understanding Across International Samples

The Promise of Machine Translation and NLP-Based Machine Learning Technologies
ByMinsu Ha, Ross H. Nehm
Size: 1.22 MB

chapter 13|18 pages

Making Sense of College Students' Writing Achievement and Retention With Automated Writing Evaluation

ByJill Burstein, Daniel F. McCaffrey, Steven Holtzman, Beata Beigman Klebanov
Size: 0.99 MB