The Blog

# Exit Poll Mania -- Beware

As a former John Kerry staffer from 2004, exit polls make me beyond anxious. On Election Day 2004, early exit polls had Kerry beating Bush, 50% to 48%. And we all know what happened.
This post was published on the now-closed HuffPost Contributor platform. Contributors control their own work and posted freely to our site. If you need to flag this entry as abusive, send us an email.

These early exit polls, courtesy of Open Left, could lead an Obama fan to faint:

Here you go. I have no idea how reliable these are.
...If these are real, Obama has thrashed Clinton today.

Georgia: Obama 75, Clinton 26

Connecticut: Obama 52, Clinton 45

Illinois: Obama 70, Clinton 29

Alabama: Obama 60, Clinton - 37

Delaware Obama 56, Clinton 42

Massachusetts: Obama 50, Clinton 47

Missouri: Obama 50, Clinton 45

Tennessee: Clinton 52, Obama 41

New York: Clinton 56, Obama 42

New Jersey: Obama 52, Clinton 47

Arkansas: Clinton 71, Obama 26

Oklahoma: Clinton 61, Obama 30

Arizona: Obama 51, Clinton 45

But, as a former John Kerry staffer from 2004, exit polls make me beyond anxious. On Election Day 2004, early exit polls had Kerry beating Bush, 50% to 48%. And we all know what happened. So as we head into the race-calling hours, with polls that will no doubt be tighter than the ones above, I'm dusting off my statistics, courtesy of my fantastic statistics teacher Deb Hughes Hallett. The horse race coverage is a dangerous thing. Take my home state in early exit polls: "Massachusetts: Obama 50, Clinton 47."

A single digit lead like this means it's too close to call, because of the margin of error. The margin of error goes like this (I found a great example online, at Robert Niles' blog, which I have adapted for this race). Say the poll has a 4% margin of error, which would be reasonable for a small sample. "That means that if you asked a question from this poll 100 times, 95 of those times the percentage of people giving a particular answer would be within 4 points of the percentage who gave that same answer in this poll.

With a 3 point lead and a margin of error of 4%, there is no reasonable statistical basis for claiming that Obama will win over Hillary in Massachusetts.

More on the MoE:

A poll of 250 votes gives the largest margin of error, about 6.2%. For a poll of 450 voters, the margin of error is about 4.6%.

What to do with the margin of error: If the difference between the two candidate's polls is less than the margin of error, you should say the race is too close to call. (Reason: The difference you are seeing could easily be the result of sampling variation in the poll.)

If the difference is more than twice the margin of error, you can say that the person who is ahead is likely to win. (In this case the difference is large enough that it is unlikely to be the result of sampling variation in the poll.)

What if the poll results are between one and two margins of error apart? This is the grey area, where it is possible that the candidate with lower poll numbers will actually win. The chance of this goes down as the difference between them gets closer to two margins of error. Let's assume that 450 people were polled: The margin of error is 4.6%. If there is a 9% difference between candidates, the top candidate probably will win. If the difference between candidates is less than 4.5%, the race is too close to call. Between 4.5% and 9%, you could to say something like "X appears to be ahead at the moment, but keep watching the polls."

Any statisticians in the audience? Am I missing anything important?

I'll be heading to Romney HQ in Boston soon -- I don't think it will be quite a party....