Over the past several days, according to the New York Times and other news sources, millions of protesters in Iran have taken to the streets to express their opposition to the official results of last week's disputed presidential election.
Earlier this week, however, the Washington Post published an op-ed by Ken Ballen and Patrick Doherty, who claimed that a scientific pre-election poll in Iran, conducted "three weeks" before the balloting (in fact, it was four weeks), presaged incumbent President Mahmoud Ahmadinejad's official landslide victory. If we can believe the poll, and the authors' arguments, it's plausible that Ahmadinejad did in fact win the election with more than 60 percent of the vote.
But don't be fooled. The same poll results can be used -- and just as persuasively -- to argue that the Iranian election was stolen, as can be used to argue the opposite viewpoint.
In fact, there are at least five reasons why we can't trust that poll. But first, some background.
What the Poll Showed
The poll was conducted from May 11-20, and incorporated what appear to be rigorous scientific sampling methods. The telephone poll (landline phones) included about 1,000 respondents, the vast majority of whom said they intended to vote. Interviewers called into Iran from outside the country and spoke Farsi. There is little to no criticism of the poll's sampling and interviewing methods.
The results showed that Ahmadinejad led his closest opponent, Mir Hossein Mousavi by 20 points - 34 percent to 14 percent. But more than a quarter of the respondents said they didn't know who they would vote for, and another 15 percent simply refused to answer.
By contrast, the official results http://en.wikipedia.org/wiki/Iranian_presidential_election,_2009show Ahmadinejad getting 63 percent of the vote to Mousavi's 34 percent, while the other two candidates together get just 3 percent.
Five (Somewhat Overlapping) Reasons Why We Can't Trust the Poll
Reason #1: The poll was conducted too far away from the election to have much predictive value.
The poll was conducted in mid-May, on average about four weeks before the election. Too much can happen in four weeks to rely on a poll this old to explain what happened on June 12.
(In the op-ed, the authors claimed the poll was conducted "three weeks" before the election, but that is disingenuous to say the least. In fact, according to their own report, interviewing was conducted over the whole length of the 10-day period (May 11-20). The poll ended three weeks and two days before the election, but the initial interviews began almost five weeks before the election (four weeks and five days). May 15-16 is the mid-point of the interviewing period, which is (on average) four weeks, not three, before the election.)
Reason #2: So many people were still undecided in mid-May, that the "2-1 lead" noted by the authors in their op-ed is close to meaningless.
The poll found Ahmadinejad getting just 34 percent of the vote. Yes, it was more than a two-to-one lead over Mousavi's 14 percent, but more than a quarter of the people (27 percent) said they were undecided, and another 15 percent refused to answer. The poll simply cannot be used as a predictor with so many undecided and undeclared.
Reason #3: The true undecided percentage was almost certainly much higher even than the 27 percent recorded in the poll.
The authors employed a "forced-choice" vote intention question, which asked "if the election were held today, who would you vote for?" Respondents were not explicitly given the option "or haven't you made up your mind?" Thus, they were pressured into coming up with an opinion, even if they didn't have one. In that case, the best known name was Ahmadinejad, and it's quite likely that many undecided voters gave his name solely because he was better known -- at that time. (Of course, 27 percent said they were undecided despite the forced-choice question, but that percentage would have been greater had the question provided an explicit "unsure" option.)
Reason #4: In general, individual pre-election polls are not reliable predictors of election results until just a day or two before the election. They are especially unreliable three and four weeks before an election.
Because most pre-election polls employ the "forced-choice" (who would you vote for if the election were held "today") question, they consistently underestimate the undecided vote and thus the potential for change. We know that polls can't be trusted weeks before an election, because reputable polls come up with contradictory estimates - until right before the election.
A recent example is the 2008 American presidential election. Just a week before Election Day, Pew Research showed Barack Obama leading by 15 points, and CBS reported an 11-point lead. At the other end of the spectrum, Investors Business Daily found Obama up by only 4 points, while the George Washington University Battleground Poll said just 3. A few days later, the latter two polls reported a surge in support for Obama, while the previous two polls showed a sudden decline, resulting in final predictions that were close to each other and to the election results.
The Iranian pre-election poll was a one-time snapshot, and a fuzzy one at that - based on its use of the force-choice question. We don't know what a similar poll would have predicted if interviewing had been conducted up close to the election. But we certainly can conclude that its results four weeks earlier are hardly relevant to what actually happened on June 12.
Reason #5: The Iranian poll report itself suggested the lead was closer than the numbers indicated, and that no candidate would achieve an absolute majority -- much less the 63 percent Ahmadinejad allegedly received.
In May, the authors of the poll report concluded, among other things, that "a close examination of our survey results reveals that the race may actually be closer than a first look at the numbers would indicate" (emphasis added). The reason: Most of those who said they didn't know which candidate they would support favored change or reform in the current political system, and thus were less likely -- according to the report -- to support the incumbent.
The report also noted that it was "likely," given the "current mood," that a run-off election would be necessary, because none of the candidates was likely to win an outright majority of the vote.
News reports of the electoral campaign from mid-May until June 12 hardly suggest the "current mood" had improved for Ahmadinejad. Indeed, one analyst suggested just the opposite, based on other polling in Iran.
Thus, it hardly seems credible for these same authors to argue, a month after their poll, that their results can be used to validate the Iranian election. Anyone who took their analysis seriously in the May report could be persuaded that the alleged 63 percent victory was in fact bogus. The campaign from mid-May to June 12 seemed to be going against Ahmadinejad, a view that seemed to be reinforced by other (though less reliable) polls, yet somehow the president was able to increase his margin from 20 points (in the May poll) to 29 points (in the official tabulation).
But I wouldn't argue that the poll can be used to challenge the election results. Instead, I believe that the most sensible position is simply to admit that the May poll cannot be used either to support or to challenge the election results.