Write a short explainer for undergraduates on what a p-value is and the most common ways people misinterpret it. Keep it between 350 and 550 words. Reply with only the finished text, with no preface or closing remarks.
Text
A p-value measures how incompatible your data are with a particular statistical model, usually one that includes a **null hypothesis**. The null hypothesis often says there is no difference between groups or no association between variables.
More precisely, a p-value is the probability of getting a test statistic at least as extreme as the one observed, **assuming the null hypothesis and the test’s other assumptions are true**. “Extreme” means farther from what the null predicts, in the direction or directions specified by the test.
Suppose researchers compare a new teaching method with a standard method. Their null hypothesis is that the methods have the same average effect on exam scores. They obtain a p-value of 0.03. This means that, if the null hypothesis and the statistical assumptions were true, about 3% of hypothetical repetitions of the study would produce a test statistic at least as extreme as the observed one.
That interpretation is narrower than several common claims:
**“There is a 3% probability that the null hypothesis is true.”**
No. A p-value describes the probability of certain data patterns given a hypothesis, not the probability of the hypothesis given the data. Calculating probabilities for hypotheses requires additional information and a different framework, such as Bayesian analysis.
**“There is a 3% chance the result happened by chance.”**
No. “Happened by chance” is too vague. The p-value evaluates results under a specific null model; it does not calculate the probability that random variation caused the observed finding.
**“A small p-value means the effect is large or important.”**
No. With a very large sample, a tiny, practically irrelevant difference can produce a small p-value. Conversely, an important effect may have a large p-value in a small or noisy study. Always examine the estimated effect size, its uncertainty, and its real-world significance.
**“A p-value above 0.05 proves there is no effect.”**
No. A large p-value means the test has not found strong evidence against the null. It may reflect a genuinely small effect, insufficient data, or substantial variability. Failure to reject a hypothesis is not proof that it is true.
**“Below 0.05 is a discovery; above 0.05 is nothing.”**
The conventional 0.05 cutoff is not a scientific boundary. Results with p-values of 0.049 and 0.051 provide very similar evidence. Nor does a small p-value guarantee that a finding will replicate.
Finally, p-values depend on how a study was conducted and analyzed. Testing many hypotheses, selectively reporting results, or repeatedly checking data and stopping when a result becomes significant can make ordinary p-values misleading unless appropriately accounted for.
Treat a p-value as one piece of evidence—not a verdict. Interpret it alongside study design, assumptions, effect sizes, uncertainty, and existing knowledge.