ABSTRACT
For many applied researchers within the social and behavioral sciences, appropriate statistical and measurement tools are selected and employed to be able to make reliable and valid inferences about substantive hypotheses under study. For other researchers, however, the methods used to practice statistics and measurement and the determination of what are the most appropriate methods is a substantive area in and of itself. To these quantitative methodologists, the research questions of interest focus more on the properties and performance of the methods themselves rather than on the methods as applied to empirical data. When possible, these methodological questions are best addressed mathematically using, for example, analytical derivations. However, when evaluating and systematically manipulating conditions such as sample size, missing data, and distributional characteristics, these analytical deviations often become intractable and alternative approaches must be employed to study the method under examination. Monte Carlo computer simulation offers one such approach. In a typical application, data are simulated to be consistent with the model structure and/or assumptions underlying the quantitative method under study; models are then fitted to those simulated data and predefined outcome measures of interest are evaluated to gauge the method’s performance. Although computer simulation methodology can offer unique and powerful insights into a broad class of quantitatively relevant research hypotheses, there exists tremendous variability in how these methods are used in practice. Sources of variability include theoretical motivation, experimental design, model selection, data generation, data analysis, and reporting of results, all of which can combine to make evaluating a Monte Carlo simulation study a challenging task. To help structure the many ways in which computer simulation might be used in practice, 12 specific characteristics are described that should typically be present in methodological research employing this investigative approach.
