WASHINGTON -- The final round of polls on Tuesday's Wisconsin recall election are something of a puzzle, although the uncertainty is more about the margin separating the candidates than the outcome. While virtually all show Republican Gov. Scott Walker leading his Democratic challenger, Milwaukee Mayor Tom Barrett, they disagree about the size of Walker's lead.
Six polls completed in the last 10 days have shown everything from a 12-percentage-point Walker lead to a dead-even race. One explanation for the difference is sponsorship: Internal polls sponsored by the Democratic campaigns have typically shown a closer contest. Beyond that, although the polls differ considerably in their methods, nothing stands out to explain why some show Walker doing better than others.
With all polls included, the HuffPost Pollster chart of publicly released Wisconsin surveys gives Walker a lead of 3.8 points (50.4 to 46.6 percent). With the polls funded by the Democratic campaigns excluded, Walker's lead grows to 6.4 points (51.7 to 45.3 percent).
In particular, several internal polls conducted by the Barrett campaign or funded by its allies have shown a considerably closer race than those sponsored by media outlets or otherwise not funded by one of the campaigns.
The difference has been evident throughout the campaign. Surveys sponsored by the Democrats and their allies have typically shown a margin about five percentage points narrower than those conducted by more independent outlets.
That difference is also consistent with the pattern of statistical bias in publicly released partisan polls, which typically skew the results a bit, boosting their sponsor by about three points and dropping their opponent by roughly the same amount.
Keep in mind that the distinction between partisan and independent can be relative. Our charts treat the Wisconsin polls by Public Policy Polling and We Ask America as independent because they lack partisan sponsors. Elsewhere, however, PPP has polled for local Democratic campaigns, and We Ask America is a subsidiary of the Illinois Manufacturers Association. The former has Walker leading by three points on its final poll, while the latter has Walker up by 12.
Can we gain any insight into the differences among the polls' results by considering their methods, particularly the way they select "likely voters"?
The Marquette University Law School polls used a very simple procedure. They selected those respondents who said they were "absolutely certain" to vote in the recall election as likely voters. The rest -- including those who described their chances of voting as "very likely" and just "50-50" and those who said they would not vote -- were considered unlikely to vote.
The Marquette pollsters also weighted their full adult sample to conform to U.S. Census estimates of the demographics of the state's adult population, including sex, age, marital status, race, Hispanic origin and education. They used live interviewers and called samples of landline and mobile phones using randomly generated numbers, which gave all working phones in Wisconsin a chance of inclusion.
The We Ask America polls used an even simpler method to select likely voters. According to We Ask America's Gregg Durham, the survey's recorded voice queried each person answering the phone if he or she was "a registered voter who plans on voting in the recall election." Those who pressed the appropriate button to say no were dropped from the survey.
We Ask America also sampled differently. It drew from Wisconsin's list of registered voters and then weighted the results from the completed interviews by gender, age, race and Hispanic ancestry. Durham said We Ask America used "the latest census data" to inform its weighting targets, but since voters differ demographically from the full population, the pollsters "kind of blend it based on the best available info."
The automated PPP surveys used a similar method. Those who answered the phone in sampled households heard a recorded voice say, "This is a short survey about Tuesday's recall election for governor. If you don't plan to vote in the recall election, please hang up now." Those who remained on the line were considered likely to vote.
PPP also sampled from the official lists of registered voters, but the firm added an extra step. It limited its selection to households with at least one voter who had actually voted in at least one of the last three general elections: 2006, 2008 or 2010. PPP also adjusted the demographics of the final sample -- age, race and gender -- using a combination of calling quotas and weighting. The weighting targets are based on "who responds to the poll, past election patterns, census," according to PPP's Tom Jensen. "It's just a combination of factors."
The Angus Reid survey made no special effort to identify likely voters: It reported results for all self-identified registered voters. The survey was very different, however, in that it was conducted over the Internet with Wisconsin voters who had previously volunteered to complete surveys as part of Angus Reid's "Springboard America" panel. Thus, only voters with Internet access who had previously joined the Angus Reid panel had a chance of being selected.
The polls sponsored by the Barrett campaign or allied campaign committees were conducted by Garin-Hart-Yang, Lake Research Partners and the Mellman Group. The polls' press releases disclosed very little methodological information and almost nothing about how likely voters were selected. The Lake Research release said it conducted an "automated survey" over the Memorial Day weekend. The Mellman release indicated that the firm sampled from a registered voter list. Neither Garin-Hart-Yang nor Lake Research responded to our requests for further information.
All of these methods involve trade-offs. The advantage of the Marquette Law School methodology is that it allows virtually all potential voters some chance of inclusion and requires no assumptions about turnout or voter demographics. Likely voters simply identify themselves. The downside is that not all of those who cast a ballot will describe themselves as "certain" to vote, and many deemed not likely to vote will cast a ballot.
In theory, pollsters who sample from registration lists improve the accuracy of their likely voter selection because they have cut out the nonregistered. They can also go the added step of limiting their sample to those with some past voting history.
Some polling experts argue that the anonymous nature of automated surveys produces a more honest answer to questions about voting intent. But the method has its own flaws, including the cost of missing the voters whose phone numbers are unlisted or cannot be matched to official records, including most of those who have cellphones but no landline service.
The bottom line here is that all of the independent Wisconsin polls used relatively simple methods of identifying the probable electorate. None followed the more complex but controversial Gallup likely voter model. If the partisan pollsters believe some aspect of their methodology is behind their closer numbers, they have not offered that explanation.
Again, the polls without partisan sponsorship put Walker ahead by an average of six points. Given that margin, as Nate Silver of The New York Times noted, "the historical accuracy of gubernatorial polls over the past 15 years" puts the odds of a Walker victory at "almost 95 percent."
Below, more on the history of the Walker recall effort:
