ABSTRACT
This is the good news chapter. In previous chapters I’ve said that, unfortunately, sampling variability is often large and CIs long. Jacob Cohen, the great statistical reformer, said “I suspect that the main reason they [CIs] are not reported is that they are so embarrassingly large!” (Cohen, 1994, p. 1002.)
The good news is meta-analysis, which can produce strong evidence where at first sight there’s only weak evidence. It can turn long CIs into short ones (well, sort of), find answers in what looks like a mess, and settle heated controversies. Much of what it does can be displayed in a beautiful picture, the forest plot, which I’ll use to explain meta-analysis and why I think it’s so great. Here’s our agenda:
■ The forest plot, the attractive face of meta-analysis ■ The simplest approach, which assumes all studies investigated the same
population: fixed effect meta-analysis ■ A more realistic approach, which assumes different studies may have
investigated different populations: random effects meta-analysis ■ Moderator analysis, which can give insight beyond an overall effect size ■ Cohen’s d for meta-analysis, when studies have used different original-
units measures ■ The seven steps in a large meta-analysis ■ Meta-analyses that have changed the world
In a single phrase, think of meta-analysis as estimation extended across more than one experiment. Here’s a slightly more formal definition:
Meta-analysis is the quantitative integration of results from more than one study on the same or similar questions.
