- A sample of ~1,000 people is statistically enough to represent 330 million Americans — margin of error is ±3pp regardless of population size
- Phone poll response rates have collapsed from 35% (1990s) to under 6% today — online matched-sample polling is now the standard
- Polls missed in 2016, 2020 and 2022 for different reasons: education weighting, differential non-response, and pollster herding — no single fix addresses all three
- Always use aggregated averages, not individual polls — they smooth out house effects and reduce the influence of outlier surveys
What Is a Political Poll?
A political poll is a survey that attempts to measure the views or voting intentions of a large population — such as all registered voters in the United States — by asking a smaller, randomly selected group of people the same questions. The logic is mathematical: a properly drawn random sample mirrors the distribution of views in the broader population within a calculable range of error.
The key word is random. A true random sample gives every person in the target population an equal probability of being selected. In practice, achieving a truly random sample of American voters is enormously difficult: some people do not have landline phones, some will not answer unknown calls, some do not use the internet, some are harder to reach in certain geographic or demographic groups. Every poll involves compromises with pure randomness, which is why weighting and methodology matter so much.
Political polls are used to measure horse-race standing (who is ahead in a head-to-head matchup), approval ratings, issue positions, primary contest dynamics, and the internal composition of candidate coalitions. They are snapshots of opinion at a moment in time, not predictions of future behavior.
Sample Sizes: Why 1,000 People Can Represent 330 Million
The counterintuitive reality of sampling statistics is that the size of the population being sampled matters very little for accuracy — what matters is the size of the sample itself. A properly drawn random sample of 1,000 people produces roughly the same margin of error whether the population is 300,000 or 300 million. This follows from the Law of Large Numbers and the mathematics of the Central Limit Theorem.
The relationship between sample size and margin of error follows a mathematical formula. For a binary question (Candidate A vs. Candidate B), the margin of error at 95% confidence is approximately:
This means a sample of 1,000 gives a MOE of roughly ±3.2pp; a sample of 400 gives ±5pp; a sample of 2,500 gives ±2pp. Doubling the sample size reduces the margin of error only by a factor of about 1.4. This is why most national polls use samples of 800-1,200: larger samples produce diminishing returns in precision while significantly increasing cost.
State-level polling typically uses smaller samples (400-600 respondents) because state populations are smaller and harder to reach. This is why state polls tend to have margins of error of ±4-5pp — making close state races especially difficult to call from polling alone.
Margin of Error: What It Actually Means
If a poll shows Candidate A at 48% and Candidate B at 44% with a ±3pp margin of error, what does that mean in practice?
The margin of error applies to each number independently. Candidate A's true support is likely between 45% and 51%. Candidate B's true support is likely between 41% and 47%. These ranges can overlap. When a poll shows a lead smaller than the margin of error, the race is within the statistical uncertainty of the poll — not a statistical tie (which is a common misconception), but a race where the uncertainty in the measurement is larger than the lead being measured.
The "95% confidence" standard means that if you ran the same poll 100 times with different random samples from the same population, 95 of those polls would produce a result within the stated margin of error. The remaining 5 would fall outside it — not because of bias, but due to random variation. This is why individual polls can produce outlier results that look inconsistent with others.
The margin of error also only accounts for random sampling error. It does not capture systematic biases in who responds, errors in weighting, problems with the question wording, or the inherent difficulty of predicting who will actually vote. The total error in a poll is often larger than the stated MOE.
Likely Voters vs. Registered Voters
One of the most consequential methodological choices in political polling is whether to survey registered voters (all people who are registered to vote) or likely voters (a subset of registered voters who are judged likely to actually cast a ballot in the specific upcoming election).
Likely voter samples almost always produce more Republican-leaning results than registered voter samples. The reason: Republican voters, particularly older voters and whites without college degrees, have higher and more consistent turnout rates. Democratic coalitions lean more heavily on younger voters and sporadic voters with lower turnout. Screening for likelihood to vote selects for Republican-aligned demographics.
The typical shift from registered-voter to likely-voter screens is 2-4 percentage points in the Republican direction. In a close race, this can be the difference between showing a Democratic lead and a Republican lead.
Different pollsters use different likely voter screens. Common approaches include asking about past voting behavior ("Did you vote in the last two elections?"), intention ("How likely are you to vote in November?"), and interest level ("How closely are you following the election?"). Some pollsters use complex multi-question screens; others use simpler single questions. These design choices explain much of the variation in results between reputable pollsters during the same period.
Phone vs. Online Polling
Phone Polling
Phone polling — calling random digits (random digit dialing, or RDD) or registered voter lists — was the gold standard for decades. The response rate to phone polls has collapsed from roughly 35% in the 1990s to under 6% today, as Americans screen calls from unknown numbers. This creates severe non-response bias: those who do answer are not a random sample. Phone polls are now expensive (a high-quality poll of 800 likely voters can cost $15,000-$30,000), making frequent polling economically challenging for most news organizations.
Online Polling
Online polls reach respondents via email panels or digital recruitment. High-quality online polling firms like YouGov use matched random sampling: they recruit a large panel, then select respondents to match the demographic profile of the target population. This approach, combined with statistical weighting, can produce highly accurate results at lower cost. The main risk is self-selection bias in who joins the panel. Online polls that simply ask anyone who visits a website to respond ("opt-in" polls) have no statistical validity and should not be treated as legitimate survey data.
Why Polls Have Missed: 2016, 2020 & 2022
2016: Education Weighting & the Hidden Trump Vote
Most national polls correctly predicted a Clinton popular vote lead but significantly underestimated Trump\'s approval in key Midwestern states. The main cause identified by the American Association for Public Opinion Research (AAPOR): surveys were not weighting properly by educational attainment. Non-college white voters, who supported Trump heavily, were underrepresented in poll samples and not corrected for through weighting. Because education correlates with political views, the samples leaned too Democratic. Some analysts also cited "social desirability bias" — some Trump supporters were reluctant to disclose their choice to an interviewer — though this was harder to verify empirically.
2020: Social Desirability Bias & Differential Partisanship
Polls substantially overstated Biden's lead: the final national polling methodology showed Biden +8; he won by +4.5. State polls in the Midwest were off by 5-7 points. The AAPOR 2020 post-election report identified differential non-response as the likely primary cause: Democratic voters — who were more politically engaged and more willing to answer surveys in the Trump era — were overrepresented in samples. Correcting for this would have required weighting by partisan identification or political interest, but both are unstable variables. The 2020 miss was larger than 2016 despite pollsters' post-2016 corrections, suggesting the problem was not simply education weighting.
2022: The Red Wave That Wasn't
Polls and forecasters widely predicted a large Republican midterm wave. Democrats lost the House but by a much smaller margin than expected, and held the Senate. In this case, the polling environment was distorted by herding: Republican-leaning pollsters released a wave of favorable surveys in the final two weeks, shifting aggregator averages toward Republicans. Several of these pollsters (including Trafalgar and Rasmussen) had a history of Republican house effects. The 2022 cycle renewed debate about how aggregators should handle partisan pollsters and whether to apply "house effect" corrections more aggressively.
What Poll Aggregators Do
A single poll is a noisy measurement. Poll aggregators — FiveThirtyEight (now ABC/538), RealClearPolitics, The Economist, Nate Silver's Silver Bulletin — combine multiple polls to reduce noise and produce a more stable estimate of where a race stands.
Simple averaging takes the mean of all recent polls. RealClearPolitics uses a rolling average of the last several polls by date. This is straightforward but gives equal weight to a poll of 400 respondents and a poll of 2,000, and does not correct for known biases.
Weighted averaging adjusts for pollster quality (as rated by historical accuracy), sample size, recency, and methodology. FiveThirtyEight's model weighted polls by pollster rating and applied house effect corrections — systematic adjustments for pollsters that historically lean Republican or Democratic. A poll from a B+ rated firm received less weight than one from an A+ rated firm.
Trend estimation uses statistical smoothing to identify the underlying direction of opinion over time, filtering out individual poll volatility. This is particularly useful in primaries where the field is large and opinion shifts rapidly.
Probabilistic models like Silver's Bulletin or The Economist's model go further: they combine poll aggregates with structural fundamentals (economic conditions, presidential approval, historical patterns) to produce win probability estimates. These models explicitly acknowledge that polling averages contain error and translate that uncertainty into probabilities.
Despite these refinements, aggregators have also been wrong — they predicted a large Clinton victory in 2016 and a large Democratic House wave in 2022. Aggregation reduces random error but cannot eliminate systematic bias when most polls in an environment share the same methodological flaw.
Polling Accuracy by Election Cycle: Where Polls Went Wrong
| Cycle | Final Polling Avg | Actual Result | Error | Primary Cause Identified | Aggregator Call |
|---|---|---|---|---|---|
| 2012 Presidential | Obama +0.7 | Obama +3.9 | ~3.2pp (D undercount fixed, overshot) | Likely voter screens too conservative; Obama coalition turnout underestimated | Mostly accurate direction |
| 2016 Presidential | Clinton +3.5 national | Clinton +2.1 | 1.4pp national; 5–7pp Midwest state polls | Education weighting; non-college white voters underrepresented | Clinton favored; wrong in key states |
| 2018 Midterms | D +8.0 generic | D +8.6 | 0.6pp | N/A — accurate cycle | D House majority called correctly |
| 2020 Presidential | Biden +8.4 national | Biden +4.5 | 3.9pp national; 5–7pp Midwest states | Differential non-response; politically engaged D voters overrepresented | Biden strong favorite; margin overstated |
| 2022 Midterms | R +2.5 generic | D +2.8 | ~5pp swing | Herding by partisan pollsters; GOP-leaning surveys flooded final 2 weeks | R wave predicted; D outperformed |
| 2024 Presidential | Harris +0.5 to Trump +0.5 | Trump +1.5 | ~2pp average | Persistent R undercount; non-response patterns; education weighting improvement | Too close to call; Trump won |
Three consecutive elections (2016, 2020, 2022) showed systematic underperformance in polling, though each had different root causes. 2018 was accurate; 2024 showed marginal improvement over 2020. The common thread: non-response bias, where certain voter groups are harder to reach, systematically skews samples. See also: Polling Methodology Explained and Approval Rating Explained.
Frequently Asked Questions
Why are polls sometimes wrong?
Polls miss for several reasons: non-response bias (certain groups are harder to reach and underrepresented), education or demographic weighting errors, likely voter screen design, social desirability bias (some voters reluctant to disclose true preference), and differential engagement where one party's voters are more willing to participate in surveys. The stated margin of error only captures random sampling error, not these systematic biases. Post-election analyses of 2016, 2020 and 2022 have each identified different primary causes, suggesting polling faces multiple overlapping methodological challenges.
What does margin of error mean?
A ±3pp margin of error at 95% confidence means that if the same poll were repeated 100 times with different random samples, 95 of those polls would produce a result within 3 percentage points of the true figure. For a candidate at 48%, the true level is likely between 45% and 51%. Importantly, the MOE applies to each candidate independently — so a 4-point lead can be within the combined statistical uncertainty. The MOE also only measures random sampling error, not systematic biases or errors in the likely voter screen.
Are online polls reliable?
High-quality online polls using matched random sampling (as used by YouGov and similar firms) can be as reliable as phone polls — sometimes more so, given the collapse in phone response rates. Simple opt-in online polls where anyone can participate have no statistical validity. The key is methodology: how the sample is recruited, how it is weighted, and how likely voters are identified. Firm-level track records (available through AAPOR and 538's historical pollster ratings) are the best guide to reliability.
What is the difference between a registered voter poll and a likely voter poll?
Registered voter polls survey everyone who is registered, while likely voter polls filter to those expected to actually cast a ballot. Likely voter screens typically produce results 2-4pp more Republican than registered voter screens, because Republican-leaning demographics (older, more habitual voters) score higher on turnout likelihood. In competitive elections, the choice between these methodologies can shift a poll from showing a Democratic lead to a Republican lead. As elections approach, pollsters typically switch from registered to likely voter samples.
