Which states have the most and least accurate polls?

Do you want to know who will win the 2024 presidential election? Flip a coin. The race will most likely come down to seven key swing states: Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania and Wisconsin. In four of those states, the average gap between former President Donald Trump and Vice President Kamala Harris in the polls is less than 1 percentage point. For the remaining three it is less than 2 points.

With numbers like that, it's easy to imagine all seven states voting for one of the two candidates. It's completely normal for polls to be off by a few points. Since 1998, general election polls for president, U.S. Senate and governor conducted within 21 days of the election have had a weighted average* error of 5.1 points. (A poll's error is defined as the difference between its margin and the final election margin. For example, a poll that gave the Republican a 5-point lead in an election that the Republican won by 1 point had an error of 4 points. )

However, this number includes national polls and polls from all 50 states. If we want to predict exactly how high the polling error will be in the swing states of 2024, we can actually be a little more precise. I calculated the weighted average error of general election polls for president, U.S. Senate, and governor conducted within 21 days of elections since 1998 in every federal state. (I've also only calculated it since 2016, which has the advantage of allowing me to zoom in on the Trump era, which is challenging for pollsters, but also risks reading too much into a smaller sample size of polls.) And it turns out that some states consistently have more accurate polls than others.

With a weighted average error of 3.3 points, the most accurate polls since 1998 have been national polls. This is not surprising for several reasons: First, the larger the population, the easier it is for a pollster to obtain a large sample size, thereby reducing survey error. Second, there are no statewide polls for Senate and governor, and polls in elections where voters did not vote have historically been less accurate than presidential polls.

Among individual states, the most accurate polls since 1998 have been conducted in Colorado, Virginia and Oregon. That's exciting for close observers of these states' politics, but it doesn't have much relevance for the 2024 election: All three are expected to easily vote for Harris.

But look at the next states in the table: Our four swing states in the Sun Belt are all among the most accurate for polling. Since 1998, Nevada's weighted average error has been 3.8 points; North Carolina and Arizona are at 4.0 points; and Georgia's is 4.1 points. And amazingly, the polls in Nevada, Georgia and Arizona were even more precisely in the Trump era. Nevada's weighted average error since 2016 is just 3.3 points. That's notable for a state widely considered difficult to poll because its population is highly transient and many Nevadans work odd hours.

In contrast, polls in the three northern swing states were not as impressive, although they are still more accurate than the average state. Since 1998, polls in Pennsylvania and Wisconsin have had a weighted average error of 4.6 points, while polls in Michigan have had a weighted average error of 4.9 points. Michigan and especially Wisconsin also performed significantly worse in the Trump era, with weighted average errors of 5.4 and 5.6 points, respectively. (Remember the 2020 ABC News/Washington Post poll that gave President Joe Biden a 17-point lead in the Badger State?) This explains why Wisconsin has also earned a reputation for being difficult to poll.

But according to the numbers, many other states are even more difficult. Eighteen states have had a weighted average error of 6 points or more since 1998. Particularly unfortunate is the pollster who has to conduct a survey in Oklahoma, Wyoming or Hawaii. They have historically had the least accurate polls of any state. In fact, polls in the Aloha State since 1998 have had a weighted average error of a whopping 10.4 points! This could be because Hawaii is a very politically distant state (it regularly has the lowest voter turnout in the country) and has unusual demographic characteristics (it is the most ethnically diverse state in the country), making it more difficult to attract the electorate there to model.

What makes it easy or difficult to survey a state? Size and demographics are definitely important factors. However, it is clear from the data that experience also matters. As the chart below shows, the states with the most accurate polls are generally also the ones that have been polled the most.

Just like your piano teacher taught you as a child: practice makes perfect. The more pollsters survey a state, the more they learn about it—which techniques to reach its voters most effectively, how to properly weight their samples, etc. This experience makes their polls more accurate.

To be clear, polls in every state are subject to at least one poll some Mistake. However, you can feel more confident polling in states where a lot of polling is conducted – which, unsurprisingly, includes most swing states. And history suggests that we should be a little more skeptical of polls in Michigan, Pennsylvania and Wisconsin than in the Sun Belt, where polls have a more accurate track record. If you're preparing for a polling error on November 5th that could affect the election, this is probably the place for you.

Footnotes

*To avoid giving prolific pollsters undue influence, the calculations in this article downweight polls from pollsters who conduct a lot of polls. Specifically, when calculating average poll error, polls are weighted by the square root of the number of polls their pollster conducted for that particular type of election in that particular cycle. For example, if Ed Begley Jr. and Associates conducted nine presidential polls in 2016 and Daniel Day Lewis University conducted one, the error of each Begley and Associates poll would receive one-third of the error weight of the Lewis University poll.