Why can't I make both α and β small just by moving the threshold?

The threshold c only decides where the two curves are cut. Moving it to the right shrinks the pink H₀ tail (α) but enlarges the orange part of the H₁ curve below c (β), and moving it left does the reverse. With the curves fixed, α and β are tied together through c and one of them always grows when the other shrinks. The only way to reduce both is to pull the curves apart relative to their width — a larger effect d, a smaller σ or, most practically, a larger n, which narrows both curves by a factor √n.

Why does the sample-size formula have the square of (z₁₋α + z₁₋β)?

For 80 % power the threshold must sit z₁₋α standard errors above μ₀ and at the same time z₀.₈ standard errors below μ₁. Those two distances add up to the gap d between the means, so d = (z₁₋α + z₁₋β)·σ/√n. Solving for n squares both sides: n = ((z₁₋α + z₁₋β)·σ/d)². For a one-sided α = 0.05 and d = 0.5σ this gives (1.645 + 0.842)²·4 ≈ 24.7, rounded up to 25; the power curve crosses 0.8 at exactly that n.

If α = 0.05, why did my 1000 experiments under H₀ give 6.1 % false positives?

α is the long-run probability of a false positive, not a guaranteed frequency in any finite batch. The number of false positives in 1000 runs is binomial with mean 50 and standard deviation √(1000·0.05·0.95) ≈ 6.9, so anything from roughly 36 to 64 (3.6 % to 6.4 %) is within two standard deviations and perfectly consistent with α = 0.05. Run the batch several times and the counts scatter around 50; the same reasoning applies to the false-negative count around 1000·β.

What changes when I switch to a two-sided test?

A two-sided test rejects H₀ whenever x̄ is far from μ₀ in either direction, so the α budget is split: each tail gets α/2 and the threshold moves out to z₁₋α/2 (1.96 instead of 1.645 at α = 0.05). For a positive effect d almost all of the power comes from the right tail, so the extra distance costs power — at d = 0.5σ and n = 20 the power drops from about 0.72 to about 0.61 — and more samples are needed for the same 80 % target. The small left tail is still counted in the power because a sample mean far below μ₀ also rejects H₀, even though it points the wrong way.

Type I & II Errors and Statistical Power

Two normal curves (H₀ and H₁), a decision threshold; α, β and power 1−β as shaded areas; effect size, σ and n sliders.

Hypothesis test

Effect size d = μ₁ − μ₀ (in units of σ)0.5

Population σ1

Sample size n20

Significance level α (sets the threshold c)

Two-sided test (reject when |x̄ − μ₀| > c)

The sample mean of n draws from N(μ, σ²) is exactly N(μ, σ²/n), so both curves are normal with SE = σ/√n and σ is treated as known (a z-test, not a t-test). H₀ puts the mean at μ₀ = 0, H₁ at μ₁ = d. Drag the white line on the canvas to move the threshold c; the α presets place it at the exact normal quantile. Φ uses Cody’s erfc (error < 10⁻¹⁵) and quantiles use Acklam’s approximation refined by Newton steps.

Truth × decision

Reject H₀Do not rejectH₀ true

Type I · α0.050

correct · 1−α0.950

H₁ true

power · 1−β0.723

Type II · β0.277

Shortcuts

Space / Enter — run 1000 experiments
R — reset

Measured values

Effect size d0.50σ

Population σ1.00

Sample size n20

Standard error σ/√n0.2236

Threshold c (reject if x̄ > c)0.3678

z of threshold (c − μ₀)/SE1.645

α = P(reject | H₀)0.0500

β = P(do not reject | H₁)0.2772

Power 1−β0.7228

n for 80 % power25

n for 90 % power35

About this model

The simulator shows the two sampling distributions that every test of a mean rests on. Under the null hypothesis H₀ the population mean is μ₀ = 0; under the alternative H₁ it is μ₁ = d, where the effect size d is measured in units of the population standard deviation σ. For a sample of n independent draws from N(μ, σ²) the sample mean x̄ is exactly normal with the same mean and standard error SE = σ/√n, so both curves are bell shapes of identical width, separated by d. The decision rule is a threshold c: reject H₀ when x̄ > c (one-sided), or when |x̄ − μ₀| > c in the two-sided variant. The pink tail of the H₀ curve beyond c is the Type I error probability α = 1 − Φ((c − μ₀)/SE); the orange part of the H₁ curve below c is the Type II error probability β = Φ((c − μ₁)/SE); the green remainder is the power 1 − β. In the two-sided case power is computed over both rejection tails, 1 − Φ((c − μ₁)/SE) + Φ((−c − μ₁)/SE). The α presets place c at the exact normal quantile c = μ₀ + z₁₋α·SE (z₁₋α/2 for two-sided); dragging c directly recomputes α and β from the same formulas. Φ is evaluated with Cody's rational approximation of erfc and quantiles with Acklam's algorithm refined by Newton steps, so the printed numbers are exact to display precision. The power curve plots 1 − β against n on a logarithmic axis for the current α and d, and the sample sizes needed for 80 % and 90 % power follow the closed form n = ⌈((z₁₋α + z₁₋β)·σ/d)²⌉. The empirical layer draws 1000 samples of size n under each hypothesis with a seeded generator, applies the threshold and reports the observed false-positive and false-negative rates alongside α and β; their scatter around the theory is itself binomial with standard deviation √(p(1−p)/1000). Simplifications: σ is assumed known (a z-test), so the Student t correction for an estimated standard deviation is omitted; the alternative is a single point value rather than a composite hypothesis; and the two-sided sample-size formula uses z₁₋α/2 while ignoring the negligible far tail. Companion simulators: statistics/normal-distribution for Φ and z-scores, statistics/central-limit-theorem for why x̄ is normal, statistics/confidence-interval for the dual view of the same quantiles, and statistics/binomial-distribution for the scatter of the empirical rates.

Who it's for: Upper-secondary and first-year university statistics (AP Statistics, A-level, introductory inference courses).

Key terms

null hypothesis
alternative hypothesis
Type I error (α)
Type II error (β)
statistical power
decision threshold / critical value
standard error σ/√n
effect size

How it works

A hypothesis test compares the sample mean x̄ with a decision threshold c: if x̄ lands beyond c we reject H₀ (μ = 0) in favour of H₁ (μ = d). Because x̄ of n draws from N(μ, σ²) is itself normal with standard error σ/√n, both sampling distributions are bell curves of the same width, shifted by the effect size d. The pink tail is the Type I error α — rejecting a true H₀ — and the orange area is the Type II error β — failing to reject when H₁ is true; the green area is the power 1 − β. Moving c trades α against β; the only way to shrink both is to make the curves narrower by increasing n or reducing σ, which is what the power curve and the closed-form sample size n = ((z₁₋α + z₁₋β)σ/d)² quantify. The model assumes σ is known (a z-test); with σ estimated from the sample the t-distribution would replace the normal.

Key equations

x̄ ~ N(μ, σ²/n), SE = σ/√n

c = μ₀ + z₁₋α · SE (two-sided: z₁₋α/2)

α = P(x̄ > c | H₀) = 1 − Φ((c − μ₀)/SE)

β = P(x̄ ≤ c | H₁) = Φ((c − μ₁)/SE), power = 1 − β

two-sided power = 1 − Φ((c − μ₁)/SE) + Φ((−c − μ₁)/SE)

n = ⌈((z₁₋α + z₁₋β) · σ / d)²⌉

Frequently asked questions

Why can't I make both α and β small just by moving the threshold?: The threshold c only decides where the two curves are cut. Moving it to the right shrinks the pink H₀ tail (α) but enlarges the orange part of the H₁ curve below c (β), and moving it left does the reverse. With the curves fixed, α and β are tied together through c and one of them always grows when the other shrinks. The only way to reduce both is to pull the curves apart relative to their width — a larger effect d, a smaller σ or, most practically, a larger n, which narrows both curves by a factor √n.
Why does the sample-size formula have the square of (z₁₋α + z₁₋β)?: For 80 % power the threshold must sit z₁₋α standard errors above μ₀ and at the same time z₀.₈ standard errors below μ₁. Those two distances add up to the gap d between the means, so d = (z₁₋α + z₁₋β)·σ/√n. Solving for n squares both sides: n = ((z₁₋α + z₁₋β)·σ/d)². For a one-sided α = 0.05 and d = 0.5σ this gives (1.645 + 0.842)²·4 ≈ 24.7, rounded up to 25; the power curve crosses 0.8 at exactly that n.
If α = 0.05, why did my 1000 experiments under H₀ give 6.1 % false positives?: α is the long-run probability of a false positive, not a guaranteed frequency in any finite batch. The number of false positives in 1000 runs is binomial with mean 50 and standard deviation √(1000·0.05·0.95) ≈ 6.9, so anything from roughly 36 to 64 (3.6 % to 6.4 %) is within two standard deviations and perfectly consistent with α = 0.05. Run the batch several times and the counts scatter around 50; the same reasoning applies to the false-negative count around 1000·β.
What changes when I switch to a two-sided test?: A two-sided test rejects H₀ whenever x̄ is far from μ₀ in either direction, so the α budget is split: each tail gets α/2 and the threshold moves out to z₁₋α/2 (1.96 instead of 1.645 at α = 0.05). For a positive effect d almost all of the power comes from the right tail, so the extra distance costs power — at d = 0.5σ and n = 20 the power drops from about 0.72 to about 0.61 — and more samples are needed for the same 80 % target. The small left tail is still counted in the power because a sample mean far below μ₀ also rejects H₀, even though it points the wrong way.

Continue with similar topics in this category — or all 10 in Statistics & Probability.

View category →

NewSchool

p-Value Explained

Launch Simulator

Test statistic on the null density; shaded one- or two-sided tail is the p-value. Why it is not the probability that H₀ is true.

NewSchool

Confidence Interval Simulator

Launch Simulator

Draw a sample, build a 90/95/99 % interval for μ; repeat 100 times and count how many intervals catch the true mean.

NewKids

Normal Distribution & z-Scores

Launch Simulator

Bell curve with μ and σ; shaded tails, the 68–95–99.7 rule, and a built-in z-score converter x → z = (x − μ)/σ.

NewFeaturedKids

Central Limit Theorem

Launch Simulator

Pick a parent (uniform, exponential, binomial, die), draw samples of size n = 1, 5, 30; the histogram of means turns normal and its spread shrinks as σ/√n.

NewFeaturedKids

Galton Board (Bean Machine)

Launch Simulator

Balls bounce left/right through rows of pegs; the bins fill a binomial that turns into a bell curve. Few rows vs many rows.

NewKids

Law of Large Numbers

Launch Simulator

Coin or die: running mean x̄ₙ for several runs converges to μ; the ±σ/√n envelope shows how slowly.

Type I & II Errors and Statistical Power

About this model

How it works

Key equations

Frequently asked questions

Related simulators

p-Value Explained

Confidence Interval Simulator

Normal Distribution & z-Scores

Central Limit Theorem

Galton Board (Bean Machine)

Law of Large Numbers

Type I & II Errors and Statistical Power

About this model

How it works

Key equations

Frequently asked questions