I’m ambivalent about statistical power. On one hand, if we’re using NHST, power is a vital part of research planning. Also, funding bodies and ethics review boards often require power calculations. On the other hand, power is defined in terms of NHST, so if we don’t use NHST we can ignore power and instead use precision for research planning, as I discuss in Chapter 13. However, I feel it’s still necessary to understand power, partly because power calculations are often required, and partly to help understand NHST and its weaknesses. I have therefore included this chapter, although I hope that, sometime in the future, power will need only a small historical mention. Statistical power has a narrow technical definition, but sometimes
“power” is used more broadly to refer to the extent that an experiment provides information to help solve our research problems, and gives us insight about the world. I’ll use the term informativeness to refer to this more general characteristic of an experiment, and I’ll distinguish it from the less important concept of statistical power. Informativeness will remain an important idea, even if we move on from NHST and no longer use statistical power. In this chapter we’ll discuss
• An introduction to power • A take-home image-the power picture • Calculating power • Intuitions about power • Post hoc power-illegitimate power • What we really want-informativeness • High power, high informativeness • Reporting power
Back in Chapter 2 I defined statistical power: It’s the probability of obtaining statistical significance-and therefore rejecting H0-if a precisely
stated alternative hypothesis is true. It’s the chance that we’ll be able to reject the null hypothesis if there’s a true effect of a particular stated size in the population.