Political polls are a controversial subject. Often, polls are used in disingenuous ways and are completely misunderstood.
If a poll is released with information that someone does not like, often that person will immediately declare that all polls are wrong and they will be proven wrong on election day. This is just not an accurate view of polling.
Polls can be useful in understanding anything from the popularity of marijuana legalization to who will control the Senate next year, but way too many people do not seem to understand the best way to read them. A lot of it is due to many in the media being absolutely abysmal at reporting on them (see more on this below). With all this in mind, here’s my primer on polls, and some general rules of thumb.
Polls are snapshots in time, not predictions: One of the most important things to consider when reading polls is when the poll was conducted. This is often the first piece of information listed in a poll, as in “this poll surveyed 1,400 likely voters September 18-21.” The poll is simply measuring what people thought, or who they supported, at that moment. I encounter a lot of people who seem to think polls are predictions. They are not.
Polls use formulas to come up with representative samples, not random calls: This is a challenge in polling, as some groups of people, especially young people, do not like to answer phones for polls. It is critical for the accuracy of a poll to use a statistical adjustment, known as weighing, to ensure it aligns with an accurate profile of the public. This is why website polls are worthless as they are not structured for the actual public but instead, the readers of that particular website.
Polling average > single poll: Due to various factors, poll results can sometimes vary a fair amount, and polling averages can give you a more accurate picture of how a race is going. An average smooths out the rough edges and a good source for them is Real Clear Politics.
Trends are important: Especially last-minute ones, and if you doubt this, just ask Hillary Clinton. If you look at a chart of the national polling in the 2016 race, you can pretty much see where the Comey letter was released. That said, be cautious…a race change from +9 in August to +4 in October may not signal the trend you think especially if there is a long timeframe between polls (in the above example, a race could have tightened to a deadheat in September but there was no poll that month, so it misses the surge in October). With this in mind…
Polling journalism often is abysmal: The reporting in late October/early November 2016 should have been about a tightening race, not a Clinton cakewalk, but for some reason journalists didn’t pick up on the clear trend in polling. The national polls showed a tightening race. On Oct. 18, Clinton had a 7.1% lead. It ended at 3.3%, just 1.2% off the final tally, Clinton at +2.1. Also, undecideds were hovering around 6%, which is never a good thing for the person with the narrow lead, and by all indications, these last-minute deciders broke hard for Trump. Many state polls showed a similar trend toward Trump. Outside of this example, there are also some terrible headlines that sensationalize polling results and misread what the polls are actually saying.
National polls are often pretty accurate: I get responses to this when I say it that are basically eyes rolling up into heads, chewing of limbs, and speaking in tongues because it goes against a “conventional wisdom,” but national polling averages have been overall pretty good for presidential elections. From 2020 going back to 2000, here is the margin of error on Real Clear Politic’s polling average vs. actual results…2020: 2.7% 2016: 1.2%, 2012: 3.9%, 2008: 0.3%, 2004: 0.9%, 2000: 0.5%. As far as generic Congressional vote, 2020: 3.7%, 2018: 1.1%, 2016: 1.7%, 2014, 3.3%, 2012, 1.4%, 2010, 2.6%.
Again, don’t bite my head off. Just sharing data, so don’t shoot the messenger if this goes against “conventional wisdom.”
State polls, on the other hand, vary: Michigan on the Real Clear Politics average was pretty good in 2020, off by a mere 1.4%. Pennsylvania? The Keystone State’s margin of error was a Mr. Blutarsky 0.0. North Carolina came in at 1.1%, Arizona at 0.6%. Meanwhile, Wisconsin continued to confound, with an error of 6%, Ohio had an error of 7.2%, and Florida had a polling error of 4.2%.
Compare poll to poll, not different polls, to see trends: As in, compare NBC/Wall St. Journal polls, not compare NBC/WSJ to Emerson. See the reason why polling averages are better than single polls, as polls can vary for the same race.
News outlet polls are not polls of their readers/viewers, but instead, sponsor the poll and have a polling firm conduct them: Out of all the myriad pet peeves I have of the spectacularly wrong social media/public reaction to polling, this is one of the tops. Fox News is not polling their viewers, so if a poll comes out that shows Biden has a 45% approval, this does NOT mean that 45% of the viewers support Biden.
Polls have important information below the “headline” data: People focus on the horserace numbers, but often polls have information buried in them that are as important. This includes demographic breakdown, voter interest, voter enthusiasm, data on particular issues, etc.
And finally, polling is neither the gospel nor silly: There are things that polling cannot do with high amounts of accuracy. Since they are snapshots of a time, they cannot predict things like how undecided voters might break (if your person is at 46-43 with 11% undecideds on the Sunday before Election Day, you need to sweat). Predicting turnout is difficult for polls to pull off as well. While people scolding young voters for not voting in October 2018 ended up looking foolish on Election Day given the record-tying 18-30 turnout because they ignored clues in polling (same in 2020), I think this year’s election is a good example. We are flying blind as far as the effects of things like the Dobbs decision and whether moderates/independents are fully grasping the threat to democracy posed by Republicans. There is also the recent underestimating of Republican support. All this said I think it is pretty clear that the claim that polling is always way off is a pearl of conventional wisdom that is based on exaggeration.
The last word, of course, will go to the voters on the first Tuesday in November, for this post, the last word goes to Willie Nelson.