Calculating Critical Values and p-Values for Exact Tests
Exact tests are well known to be efficient and reliable statistical tools in a variety of applications. The statistical literature has extensively addressed many parametric, semiparametric, and nonparametric exact tests that control Type I error rates given finite sample sizes. In certain scenarios, exact tests control the Type I error rate much less than the desired error rate due to the fact that exact tests have a finite number of error rates given a fixed sample size. This chapter introduces methods that can be used to calculate the critical values of exact tests, including three optional procedures. These procedures include: a traditional technique based on Monte Carlo (MC) evaluations, an interpolation method based on a regression technique that uses tabulated critical values of the test statistic, and a Bayesian-type method that uses tabulated critical values of the test as prior information and MC-simulated test statistic values as data.