For nearly two decades the United States Medical Licensing Examination has included computer-based case simulations that require examinees to provide care for simulated patients. This format presents a dynamic simulation of the patient-care environment in which the patient’s condition changes based on both the underlying problem and the actions taken by the examinee. The simulations are scored using an automated scoring system. In this chapter we describe the historical background that led to development of the simulations and the subsequent refinement of the construct that occurred as the interface was being developed. We then describe the evolution of the automated scoring procedures from linear regression modeling to rule-based procedures. We conclude the chapter with a discussion of the practical challenges involved in large-scale high-stakes use of automated scoring for licensure.