Poll-er Opposites

We all knew this day would come. The day when I had to talk about polls. Believe me, I was dreading it as much as you were.

Before I go into everything that's wrong with election polling, and which polls you should and shouldn't trust, I'll say a few words about what's good about public opinion polls. In three recent major elections (2008, 2010 and 2012) the polls predicted election results fairly accurately. In this election season, public opinion polls were showing the rise of Trump before anyone in the mainstream thought of his candidacy as legitimate. The polls did something that even light of my life Nate Silver didn't do, and predicted that Donald Trump would win the Republican nomination.

Public Opinion Polls: More Boring than the Festivus Pole

So if the polls are more accurate than Nate Silver, what do I have to complain about?

Here's the very important distinction I want to draw with this blog post: Many polls, all aggregated together, accounting for biases and other issues, are mostly accurate and predictive of elections. One single poll that shows Jill Stein winning against Gary Johnson, with Trump and Clinton left in the dust, is not.

Not all polls are created equal. Some polls have demonstrated their accuracy over many years, and quickly adapt to new forms of technology, while not having a significant bias for one party. Other polls have measurable biases towards one party or another, which has lead them to be less accurate. I'm not going to name names, but one example of this type of poll starts with an R and ends with -asmussen. It's the Rasmussen poll. Stop sharing articles that use that poll as "proof" that Hillary Clinton will lose to Trump. It's not a good poll.

This headline probably just used the Rasmussen poll and called it a day, which shows me that news reporting hasn't changed much.

What makes some polls more accurate than others? The best polls are the ones that are able to mimic what the electorate actually looks like. A poll where only white people are surveyed wouldn't be an accurate poll because our elections are not limited to white people (anymore, though it did take awhile). However, a lot of polls fall into the trap of not diversifying their sample. This can happen either because they're saving money by only calling voters who voted in past elections, and if you narrow your sample to only people who voted in the midterms, you're going to be calling a bunch of senior citizens and me, which is not a reliable predictor of the electorate.

Another problem polls run into is that many polls only call landlines. This is because a 1991 law forbids people from using auto-dialing software to call cellphones. The poor pollsters don't want to have to dial every number by hand, so some polls choose to just use the software and only call landlines. And the only people who still answer their landlines are old people.

To control for skewed demographics, polling firms will sometimes mathematically weight the results. We are wading into a very technical territory that I may not have the skills to explain, but just put on your rainboots and we will get through this together. As far as I can tell, pollsters will sometimes give answers from certain demographic groups more sway in the final calculations. If they only speak to one Native American, they may weight that result so that, instead of it being .5% of the total sample, it is 2% of the total sample, which better matches the percentage of Native Americans in the country. But using faulty demographic targets (like the 2010 census, when our demographics are already shifting) or limiting how much they can weight a single answer, can sometimes make the poll less effective.

I have no idea how they weight the answers or what math goes into it, because I managed to go through my life without ever taking a statistics class. Rather than go more in depth, I'm just going to trust the experts to tell me which polls are worth listening to. FiveThirtyEight has a list where they have graded many of the major polls, factoring in how often they predict the election and what their biases are. Next time you see an article that talks about public opinion polls, you can pull up this list to see which polls they are talking about, and if those are even reputable polls. Then you can decide for yourself (with the help of my future best friend Nate Silver) whether or not Gary Johnson actually has a chance in this election, or if he's going to have to slink back to New Mexico like he did in 2012.

Once again, Nate Silver saves the day (and reminds us that Gary Johnson is a wingnut)