ABSTRACT
Understanding the meaning of p-values is a fundamental, interesting, yet broad and complex topic. It relates to how most scientists view the strength of an experiment, including its repeatability (or replicability or reproducibility; see Chapter 2, Section 2.3) and, in turn, creditability. However, there are many misunderstandings or misuses of p-values. The American Statistical Association (ASA) on March 7, 2016, has released a “Statement on Statistical Significance and p-Values” with six principles underlying the proper use and interpretation of the p-value. The intention was “to improve the conduct and interpretation of quantitative science and inform the growing emphasis on reproducibility of science research.” (Wasserstein and Lazar 2016, with discussion). The statement’s six principles are assertions (without elaboration). As we know, almost all clinical trials reported in medical journals use p-values as an indicator of evidence and summary of the results. Thus, appreciating what the p-value means and does not mean, and how it should be used and interpreted is an important subject for researchers and practitioners in clinical trials. In the following, we first highlight the ASA’s six principles and then discuss more details so that these principles, as assertions, are understood and practiced better. The six principles are as follows:
P-values can indicate how incompatible the data are with a specified statistical model.
P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
Proper inference requires full reporting and transparency.
A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.
