Confidence Intervals and What the Margin of Error Means

A poll reports that 54% of voters support a candidate, with a margin of error of plus or minus 3 percentage points at the 95% confidence level. Most readers interpret this as: there is a 95% probability that the true support is between 51% and 57%. That is not what it means. The correct interpretation is more subtle, and getting it right changes how you use these intervals.

Building the Interval from First Principles

Suppose the true proportion of voters who support the candidate is p (unknown). You survey n voters and find a sample proportion p-hat. By the central limit theorem, p-hat is approximately normally distributed with mean p and standard error sqrt(p(1-p)/n).

A 95% confidence interval uses the fact that 95% of a normal distribution falls within 1.96 standard deviations of the mean. The interval is:

p-hat +/- 1.96 x sqrt(p-hat(1-p-hat)/n)

For a poll with n = 1,000 and p-hat = 0.54:

SE = sqrt(0.54 x 0.46 / 1000) = sqrt(0.000248) = 0.0158

Margin of error = 1.96 x 0.0158 = 0.031, or about 3 percentage points.

What 95% Actually Refers To

The 95% refers to the procedure, not to any single interval. If you repeated this poll many times, each time drawing a fresh random sample of 1,000 voters, computing the interval the same way each time, then 95% of those intervals would contain the true value of p.

Once you have one specific interval, say (51%, 57%), that specific interval either contains the true p or it does not. There is no probability to assign. The true proportion is a fixed number, not a random variable. What varies from sample to sample is the interval, not the truth.

This is not pedantry. It affects how you communicate results. Saying "there is a 95% chance the true value is in this interval" implies the true value is uncertain and random. The correct framing is "this method produces intervals that capture the true value 95% of the time."

What Changes the Width of the Interval

Three things affect width. Confidence level: a 99% interval is wider than a 95% interval because you are requiring the procedure to succeed more often. Sample size: doubling n shrinks the margin of error by a factor of sqrt(2), roughly 1.41. Variability in the data: proportions near 0.5 produce the widest intervals because p(1-p) is maximized at p = 0.5.

This last point explains why the "worst case" margin of error reported in polls often uses p = 0.5 even when the actual proportion is different. It gives a conservative upper bound on the width.

Confidence Intervals versus P-values

A confidence interval and a hypothesis test are two sides of the same coin. If a 95% confidence interval for the difference between two groups does not include zero, the corresponding two-tailed test at alpha = 0.05 would reject the null of no difference. If the interval does include zero, you would fail to reject.

But the interval is more informative. It tells you the range of plausible effect sizes, not just whether an effect is detectable. A significant result with a confidence interval of (0.001, 0.002) suggests a real but tiny effect. A significant result with a confidence interval of (2, 40) suggests a large effect with considerable uncertainty about its exact size.