ABSTRACT
The chapter addresses three of the statistical indicators of the importance, or significance, of inferential statistical tests: statistical significance, effect size, and confidence intervals. Inferential statistics is all about using sample data to reach conclusions (i.e., make inferences) about the phenomenon in the population. First, a discussion of statistical significance is presented. This discussion includes a consideration of how to determine the probability that a result found with sample data (e.g., the difference to sample means) may have occurred by chance, or random sampling error. When this probability is small enough (e.g., p < 0.05), the result is considered statistically significant. However, there are limitations to tests of statistical significance, including the large effect of sample size on the p value. Next, the chapter discusses the idea of hypothesis testing and Type I and Type II errors. The logic and drawbacks of hypothesis testing are explained and illustrated with examples. Because of the limitations of statistical significance and hypothesis testing, researchers are increasingly turning to measures of effect size, and effect size is what the chapter describes next. Cohen's d and the issue of practical significance are described in some detail. Finally, confidence intervals are described. The chapter includes a discussion of the value of confidence intervals, how to calculate a confidence interval for the mean, and how to interpret it. The chapter concludes with an example that illustrates how to apply and interpret a test of statistical significance, effect size, and confidence interval of the mean in a single one-sample t test.
