The Same Question, Many Samples
A statistic like the sample mean changes from one sample to the next, and that predictable spread of estimates is measured by the standard error. · 12 min
Take a fair random sample, compute its mean, and you get a number. Take another random sample of the same size and you get a slightly different number. Neither is wrong. A sample statistic is itself a quantity that varies from sample to sample — and the remarkable thing is that this variation follows a predictable, measurable pattern.
Guess before you learn
A population has a mean of 100. You take many random samples of size 25 and record each sample's mean; those means scatter around 100. If instead each sample had size 100 — four times larger — the scatter of the sample means would be about how many times as wide?
Quadrupling the sample size cuts the scatter in half, not to a quarter. The spread of sample means shrinks with the square root of n: √4 = 2, so the width becomes 1/2. This square-root law is the single most useful fact about sampling variability.
9–12
3–5
Every sample gives a slightly different answer. If you wrote down the mean of many samples, those means would make their own little pile — bunched near the true value, with a few farther off. Bigger samples make a tighter pile. The width of that pile is called the standard error, and it tells you how much your one sample's answer is likely to wobble away from the truth.
6–8
Because a sample is chosen by chance, any statistic computed from it — the sample mean, say — changes from sample to sample. Collect the means of every possible sample and they form a distribution of their own, the sampling distribution. It centers on the true population value, and its spread measures how much a single estimate typically misses by. That spread is the standard error: for a sample mean, SE = σ / √n, the population standard deviation divided by the square root of the sample size. Larger samples give smaller errors — but only through the square root, so halving the error takes a fourfold increase in sample size.
9–12
Treat the sample mean x̄ as a random variable. Over repeated random samples of size n from a population with mean μ and standard deviation σ, x̄ has mean μ — it is centered on the truth — and standard deviation σ/√n. That last quantity is the standard error: the standard deviation not of the data, but of the estimate. The √n sets the rate of improvement: quadruple n to halve the error, multiply n by 100 to shrink it tenfold. And by the central limit theorem, for reasonably large n the sampling distribution of x̄ is approximately normal whatever the population's shape — so the 68–95–99.7 rule applies, and about 95% of sample means land within 2 standard errors of μ.
K–2
Scoop a cup of jellybeans from a big jar and count the reds. Scoop again and you get a slightly different count. Small scoops jump around a lot. Big scoops give steadier, closer counts every time.
Undergrad
For an i.i.d. sample, Var(x̄) = σ²/n, so SE(x̄) = σ/√n; the estimator is unbiased, with variance vanishing at rate 1/n. In practice σ is unknown and replaced by the sample standard deviation s, giving the estimated standard error s/√n — the extra uncertainty that motivates the t distribution at small n. The standard error, not the sample standard deviation, is the correct measure of an estimate's precision. The central limit theorem supplies the shape: x̄ is asymptotically normal for any finite-variance population, faster for less skewed ones. Two caveats: the 1/√n rate assumes independence — clustered or autocorrelated data inflate the true error — and a finite-population correction applies when the sample is a large fraction of the whole.
Postgrad
The standard error is the standard deviation of an estimator's sampling distribution — a frequentist object indexed by the design and the estimand. For x̄ under i.i.d. sampling it is σ/√n; in general it is √(Var(θ̂)), estimated by the plug-in, the delta method, the jackknife, or the bootstrap when no closed form exists. Confusing the standard error with the population standard deviation is among the most common errors in applied work: one describes the estimate, the other the data. The 1/√n law is fragile exactly where it is most assumed — positive intra-cluster correlation ρ inflates the variance by a design effect of about 1 + (m − 1)ρ, so a large sample of correlated units can carry the information of a far smaller independent one.
standard error
The standard deviation of a statistic across samples — how much a single estimate typically differs from the true value. For a sample mean, SE = σ/√n.
A population has standard deviation σ = 20. Find the standard error of the mean for a sample of size 100, then for size 400. — the steps fade as you master them
SE = σ / √n
SE = 20 / √100 = 20 / 10 = 2
SE = 20 / √400 = 20 / 20 = 1
Four times the sample size, half the standard error
There is more: not only does the standard error shrink predictably, the sample means also arrange themselves in a familiar shape. For a large enough sample, the distribution of the sample mean is approximately normal — even when the population it came from is not. That means the 68–95–99.7 rule applies to your estimates, so about 95% of sample means fall within two standard errors of the true value.
A single sample gives a single estimate, and the standard error tells you how much to trust it: how far, typically, it sits from the truth. That number is the raw material of the next step. Attach a margin to your estimate built from the standard error, and you have a confidence interval — an honest statement of what one sample can and cannot claim.
Practice — new ink and old, interleaved
1.Which statement about the sampling distribution of the mean is correct?
2.A population has standard deviation σ = 50. Find the standard error of the mean for a sample of size 100.
3.For an approximately normal sampling distribution, about what percent of sample means fall within 2 standard errors of the true mean? Enter a whole number.
4.A simple random sample of size 50 means:
5.A histogram has one tall bar on the left and a long tail stretching to the right. Its shape is:
6.In one sentence, why does the same 68–95–99.7 rule work for both heights and test scores, even though their units differ?
7.A perfectly computed standard error still cannot rescue an estimate from a voluntary-response sample, because:
8.For a fixed population standard deviation, order these sample sizes from the largest standard error to the smallest.
- n = 25
- n = 100
- n = 400
9.Which pair of measures both describe spread rather than center?
10.About what percent of a normal distribution lies within one standard deviation of the mean?