What Is a P-Value? How to Calculate It with Examples

Search Fintrev

Search our guides, reviews and glossary.

A p-value (probability value) is the probability of getting a test result at least as extreme as the one you observed, assuming the null hypothesis is true. It measures how surprising your data would be if nothing unusual were going on. A small p-value means the data would be unlikely under the null hypothesis, which counts as evidence against it. A large p-value means the data fit comfortably within what the null hypothesis predicts.

In statistics, the p-value also equals the smallest significance level at which you would reject the null hypothesis. If a test returns p = 0.03, you would reject the null at the 5% level but not at the 1% level.

How a P-Value Works

Every hypothesis test starts with two competing statements:

• Null hypothesis (H₀): the default claim, usually “no effect” or “no difference.” Example: a trading strategy’s average return per trade is zero.
• Alternative hypothesis (H₁): the claim you want evidence for. Example: the average return is not zero (two-tailed), or is greater than zero (one-tailed).

You collect a sample and summarize it in a test statistic, such as a z-score or t-statistic. The test statistic measures how far the sample result sits from the value the null hypothesis predicts, in units of standard error. The p-value then answers one question: if H₀ were true, what share of possible samples would produce a test statistic this far out or farther?

One-tailed vs. two-tailed tests

• Upper-tailed test (H₁: parameter is greater than the reference): p = P(statistic ≥ observed value)
• Lower-tailed test (H₁: parameter is less than the reference): p = P(statistic ≤ observed value)
• Two-tailed test (H₁: parameter is different from the reference): p = 2 × P(statistic ≥ |observed value|)

The same data give a two-tailed p-value twice the matching one-tailed p-value. Choose the direction before seeing the data; switching to a one-tailed test afterward halves the p-value without adding evidence.

How to Calculate a P-Value

In practice, p-values come from statistical software, spreadsheet functions or distribution tables. Mathematically, a p-value is the area under the test statistic’s probability distribution covering every value at least as extreme as the observed one.

The general steps

1. State H₀ and H₁, and decide whether the test is one-tailed or two-tailed.
2. Choose a significance level (α) in advance, such as 0.05.
3. Compute the test statistic from your sample.
4. Find the tail probability for that statistic from the relevant distribution (normal, t, chi-square, F) using the correct degrees of freedom.
5. Compare p to α. If p ≤ α, reject H₀. If p > α, do not reject H₀.

P-value formula for a mean

For a test on a sample mean when the population standard deviation (σ) is known, the test statistic is a z-score:

z = (x̄ − μ₀) / (σ / √n)

where x̄ is the sample mean, μ₀ is the mean under the null hypothesis and n is the sample size. The z-score follows the standard normal distribution, the familiar bell curve, so the two-tailed p-value is 2 × P(Z ≥ |z|).

When σ is unknown and estimated from the sample standard deviation (s), which is the usual case, the statistic becomes:

t = (x̄ − μ₀) / (s / √n)

This t-statistic follows a t-distribution with n − 1 degrees of freedom. Degrees of freedom shape the distribution: with small samples the t-distribution has fatter tails than the normal curve, so the same statistic produces a larger p-value. As n grows, the t-distribution converges toward the normal.

Reference points for the standard normal distribution:

How to find the p-value in linear regression

Regression output reports a p-value for each coefficient. The null hypothesis for each one is that the true coefficient equals zero (the variable has no linear effect). The test statistic is:

t = coefficient estimate / standard error of the coefficient

It is compared against a t-distribution with n − k − 1 degrees of freedom, where k is the number of predictors. A coefficient with a p-value below your chosen α is called statistically different from zero. This is the standard way p-values appear in econometrics, for example when testing whether an interest-rate differential helps explain currency returns.

Worked Example: Testing a Backtested Strategy

Hypothetical example for illustration only.

A trader backtests a forex strategy over 100 trades. The average return per trade is +0.15%, with a sample standard deviation of 0.6%. The question: is that average distinguishable from zero, or could a strategy with no edge produce it by chance?

• H₀: mean return per trade = 0
• H₁: mean return per trade ≠ 0 (two-tailed)
• α: 0.05, chosen in advance

Step 1: standard error. 0.6% / √100 = 0.6% / 10 = 0.06%

Step 2: test statistic. t = (0.15% − 0) / 0.06% = 2.5

Step 3: p-value. With 99 degrees of freedom, a two-tailed t of 2.5 gives p ≈ 0.014. (Using the normal approximation, 2 × P(Z ≥ 2.5) ≈ 0.012.)

Step 4: decision. 0.014 < 0.05, so the trader rejects H₀ at the 5% level.

If the strategy truly had a zero average return, 100 trades would show an average this far from zero (in either direction) only about 1.4% of the time. That is evidence against “no edge,” not proof the strategy will be profitable live, and it relies on the independence and distribution assumptions discussed under limitations.

Now change one input. If the same +0.15% average came from only 25 trades with the same 0.6% standard deviation, the standard error becomes 0.12%, t = 1.25, and the two-tailed p-value rises to roughly 0.22. Same average result, much weaker evidence, purely because of sample size.

Interpreting a P-Value

Is a p-value of 0.05 significant?

By the most common convention, a p-value of 0.05 or lower is treated as statistically significant, and the null hypothesis is rejected. Above 0.05, the result is described as not statistically significant and the null hypothesis is not rejected. Some fields use stricter cutoffs such as 0.01, and some use 0.10. The U.S. Census Bureau, for example, requires that analyses with a p-value greater than 0.10 state that the difference is not statistically different from zero.

“Not rejecting” H₀ is not the same as proving it. A p-value of 0.30 means the data are consistent with the null hypothesis, not that the null hypothesis has been confirmed.

What does a p-value of 0.001 mean?

A p-value of 0.001 means that if the null hypothesis were true, results at least as extreme as yours would occur about once in 1,000 samples. Either you observed a rare outcome or the null is wrong, so the null is rejected at any conventional significance level. A reported p < 0.001 means the value fell below that boundary.

Comparing results with p-values

Reporting the p-value itself, rather than just “significant” or “not significant,” lets readers apply their own threshold. If a test on return data gives p = 0.08, an analyst using α = 0.10 (90% confidence) calls the difference significant while one using α = 0.05 (95% confidence) does not. The p-value shows exactly where the evidence sits.

P-values also rank evidence across comparable tests. A p-value of 0.001 is far stronger evidence against the null than 0.04, even though both clear the 0.05 bar. If portfolio A differs from a benchmark with p = 0.10 and portfolio B with p = 0.01, the evidence that B genuinely differs is considerably stronger. Such comparisons hold only when the tests use similar data, sample sizes and assumptions.

What a P-Value Is Not

• P-value vs. probability the null is true. A p-value of 0.03 does not mean there is a 3% chance H₀ is true. It is computed assuming H₀ is true, so it cannot measure H₀’s probability.
• P-value vs. significance level (α). α is the threshold you set before the test, and it is the false-positive rate you accept. The p-value is calculated from the data afterward.
• P-value vs. confidence level. The confidence level is 1 − α, so 95% confidence corresponds to α = 0.05. A p-value of 0.08 would be significant only at confidence levels of 92% or lower.
• Statistical vs. practical significance. With a large enough sample, a trivially small effect produces a tiny p-value. A strategy averaging 0.001% per trade over millions of trades could be “highly significant” yet useless after spreads and commissions.
• Low p-value vs. proof. Even a very small p-value leaves some chance the result came from noise. Repeated studies or out-of-sample testing are what build confidence in a relationship.

Limitations of a P-Value

Multiple testing. At α = 0.05, testing 20 strategies that have no real edge will, on average, flag one as significant by chance. Running many backtests, parameter combinations or indicator settings and reporting only the best one makes its p-value misleading. Corrections such as the Bonferroni adjustment (dividing α by the number of tests) or holding out untouched data for a final check reduce this problem.

Distribution assumptions. The z- and t-tests assume roughly normal sampling distributions and independent observations. Market returns often have fat tails, skewness and volatility clustering, and consecutive trades may not be independent. When the assumptions fail, the reported p-value can be too small or too large.

Sample size cuts both ways. Small samples can hide real effects, as the 25-trade version of the worked example showed, and huge samples can make negligible effects look significant.

Arbitrary cutoffs. Treating 0.049 and 0.051 as categorically different overstates what the numbers show. They represent nearly identical evidence.

No information about magnitude. A p-value shows whether an effect is distinguishable from zero, not how large it is. Pair it with the effect size and a confidence interval.

The Bottom Line

Traders and analysts testing strategies or regression results can use p-values to judge whether a result stands apart from chance. A p-value does not measure the probability that the null is true, and it says nothing about effect size or live profitability. Report the exact value, pair it with an effect size and confidence interval, and confirm findings with out-of-sample data, especially after running many backtests.