ABSTRACT
Multilevel modeling, which you have likely been introduced to through hierarchical ANOVA, provides the opportunity to examine clustered or nested data. The study of individuals within organizations, such as students within classrooms and schools, employees within companies, patients within hospitals, and residents within neighborhoods, are all common examples of clustered data. In many situations, researchers are interested not only in modeling factors at the individual level but also at the group level (e.g., classroom, organization) because context-at least in education and the social and health sciences, as well as many other disciplines-is important. These higher-level factors are contextual effects that may (and usually do) contribute to or influence the individual outcomes being studied. All of us have been involved in many different types of group settings. It is probably easy for you to imagine, and you have likely experienced this for yourself, how the context of the group (e.g., the personalities of the other individuals in your group, the resources available to you, the physical environment) can greatly influence your experience in that setting. For example, the school level percentage of students on free and reduced lunch may help explain the outcomes of students within schools and the median housing price may help explain outcomes of residents within neighborhoods. Even if the interest is not in contextual effects, the hierarchical or clustered nature of the data must still be accounted for in the modeling, as nested units are more likely to produce similar outcomes as compared to units that are randomly sampled. Ignoring the clustering (assuming independence of errors) most often produces estimates with incorrect and decreased standard errors, because it is usually the case that clustered data violates the assumption of independent and identically distributed errors. Think back to the very simple one-sample t test, probably the most elementary inferential procedure you know. The denominator of the t test statistic formula is the standard error. Holding all else constant, as the standard error decreases, the test statistic increases. As the test statistic increases, the likelihood of rejecting the null hypothesis increases. This same concept applies with other, more advanced statistical procedures. The point is that ignoring clustering and hierarchical nature of data will usually result in the increased probability of a Type I error-rejecting the null hypothesis when it is really true. This translates to interpretations suggesting evidence of real effects/differences/relationships, when in fact the results are simply an artifact
of failing to account for the clustered design of the data. This is a good time to bring up the concept of ecological fallacy, interpreting relationships that are observed in groups to hold also for individual units (e.g., Freedman, 1999), and atomistic fallacy, interpreting relationships that are observed in individual units to hold also for groups (Hox, 2002). These fallacies are “a problem of inference, not of measurement” (Luke, 2004, p. 6, emphasis original), and can be combated with multilevel modeling.
