ABSTRACT
This chapter has nine sections. Section one begins with a brief introduction about the use of observational methods for personnel assessment in the health professions. We point out that assessments based on observational data are used widely in health professions education yet the quality and utility of these assessments is rarely gauged. We also assert that observational assessment is chiefly formative, a type of dynamic testing (Grigorenko & Sternberg, 1998) where learner assessment and instruction coalesce. Section two addresses the purpose and focus of observational assessments. It answers two questions. “How can we describe the clinical performance of learners?” [and] “Are the behaviors we observe in learners similar to the behaviors needed for patient care?” Section three focuses on the social character of observational assessment. Here we assert that health professions education and assessment is an interpersonal enterprise, especially when done in clinical settings. Interpersonal behavior can yield accurate data yet always has room for subjectivity, selective perception, and measurement error. Observational assessments need to reduce these sources of bias. Section four presents an observational assessment toolbox as a table. Here we show how assessment goals and tools should be matched and identify advantages and problems of these pairings. Section five covers the acquisition of observational assessment tools either “off the shelf,” from donation or purchase, or by means of constructing new measures. “Off the shelf” acquisition of measurement tools requires a hefty dose of caveat emptor [let the buyer beware]. Constructing new observational measures is hard work, yet yields
large and lasting benefits if done correctly. Section six offers tips about how to administer an observational assessment, especially about standardization and control. Section seven is about data quality, arguing that data are useless for any purpose if their reliability and validity are suspect. Section eight talks about how to use observational assessment data toward the formative goal of learner improvement and also addresses mastery learning. Section nine reiterates a set of practical recommendations published earlier about how to improve the quality and utility of observational assessments in health professions education. There is a coda.
