Can You Rig A Presidential Poll?

"rigged: manipulated or controlled, usually by deceptive or dishonest means"

According to Merriam-Webster (the dictionary company), "RIGGED" is one of the top political buzzwords of 2016. In this election cycle, it seems candidates (and their surrogates) claim everything is rigged: primary races, allocation of delegates, debate schedules, media coverage, and (near and dear to my heart), the polls.

As a statistician, if I'm trying to see if a poll is rigged, I look for what we call "bias" in these polls. Bias is the notion that estimates from these smaller samples (thousands of people) differ from the views of the larger population of all voters (millions of people) in a systematic way that is being driven by the pollster's choices, methods, or misinterpretation.

So how could someone rig an election poll? And, what can you, as a consumer of polling, look for to make sure you aren't being misled?

To answer that question, you need to understand how polling works. The goal of a poll is to understand the voting patterns of the larger voting public -- 130 million or more voters. Since the only "poll" where everyone participates is Election Day, pollsters look at smaller groups of people to determine the state of the race. Most people don't realize that we only need to look at a relatively small sample of potential voters -- perhaps a few thousand -- to potentially predict the voting patterns of the larger population. (As Arthur Nielsen Sr. -- founder of the ACNielsen company -- reportedly said, "If you don't believe in sampling, the next time you have a blood test, ask them to take it all out.")

Systematic bias can come about for several reasons, but here are a few ways you could rig a presidential poll:
  • Ask leading questions: The way a question is asked in a poll can influence the results. For this reason, many polling organizations not only field test their questions before, but once they settle on a given set of answers, they largely keep them consistent over time. But the order of questions, and whether certain negative or positive issues are addressed before or after a series of questions, can also influence the results. This type of "conditioning" is one way in which polls could be biased. What you can do: Every reputable polling organization now makes their questions available. You can look at the surveys yourself to spot places where questions may be poorly worded or confusing, or where conditioning may be occurring.

  • Don't let people choose "none of the above": Because many surveys are multiple choice, the answers can also influence the results. For example, removing the option of "none of the above" or "undecided" in a question comparing Clinton and Trump can lead to very different answers. Why? Because this may force an undecided voter to make a choice when they really aren't ready to. (In fact, a controversy has recently erupted over one polling organization, Reuters, that decided to drop "neither" from its set of answers.) What you can do: In this election, one critical alternative choice will be the inclusion or exclusion of third-party candidates. Watch for changes in the results with the inclusion or exclusion of other candidates.
  • Ask the wrong voters: If you are trying to represent the voting population of the entire country with only a few people, who you select really matters. Even with new polls coming out almost daily, by my calculation only about 73,000 people have been sampled in the major national polls in July and August. That's less than 1 percent of the expected voting population. More people fit in Lambeau Field for a Green Bay Packers game. So, who the people in the survey are matters a lot. If the sample is not representative -- for example, biased toward more Republicans, or more veterans, or any other direction -- the results will be skewed. Polling agencies spend a lot of time both trying to select representative samples, and then weighting their results so they are representative of the broader population. What you can do: Every reputable poll reports the percentages by demographic group in the poll. You can see the percentage of Democrats, Republicans, independents, men, women and other groups. It's a simple way to understand the polling.
  • Only focus on today's poll: Perhaps the place where people can be most misled is in interpreting the polls -- or, rather, in over interpreting them. The news media likes to look for narratives, but as result, it is easy to both over interpret any given snapshot, and to misunderstand what a polling result may or may not mean. Margin of error is a measure of statistical uncertainty -- but it doesn't capture all of the other types of biases above. In fact, one of the reasons that the current movement in the polls toward Clinton is interesting is the numbers have moved outside the margin of error across the board -- meaning now we are seeing results in which we can have statistical confidence in. What you can do: Treat each poll as what it is -- a snapshot of the current state of the race. A margin of error can tell you whether we can statistically distinguish results in a given poll, but it does not measure all of the other inherent biases.
  • So, can you rig a presidential poll? If by rig you mean a vast conspiracy or systematic effort to coordinate every poll in favor of one candidate or another, the answer is no, probably not. What is more likely though -- and what we've seen throughout this election cycle--is misinterpretation of certain polling results and overemphasis on key results without context.

    If you're looking for rigging in the polls, these are the issues you need to consider.

    John H. Johnson, PhD is CEO of Edgeworth Economics and the author of "EVERYDATA: The Misinformation Hidden in the Little Data You Consume Every Day."