Fox News released a poll on Thursday showing a mere 3-point lead for Barack Obama, and McCain officials have apparently pushed the results around to some reporters, trying (again) to stir up a narrative of a GOP comeback.
Two points: 1) the Fox poll isn't evidence of a McCain comeback. And 2) contrary to some speculation online, it's highly unlikely that Fox News was trying to engineer a closer result in order to drive coverage about the race "tightening."
Though Andrew Sullivan and Marc Ambinder have correctly pointed out that the partisan balance of Fox's poll is a good reason to discount its overall reflection of voter intentions -- since it appears too many Republicans were interviewed -- there's no reason to believe that Fox came by its data via anything but traditional polling methodology.
The devil is in the verbs. "Sampling" is the process of selecting and getting responses from polled individuals. There's chance involved in this. In conducting a random sample, sometimes more women respond to the calls or interviews than men. Sometimes more Democrats respond than Republicans. "Weighting" is a pollster's term for taking a random sample of respondents and then adding additional weight to certain individuals in order to bring them proportionally in line with the broader population. Gender, race and age are examples of random variables that many pollsters "weight."
But partisan ID is not traditionally weighted by most pollsters. The Huffington Post wrote about this practice earlier this year -- during a span in which a great number of polls seemed to have over-sampled Republicans without correcting for the data on the country's partisan makeup via "weighting."
John Zogby, something of a polling maverick, weights his polls to achieve a steady partisan balance from survey to survey. Gallup, by contrast, does not weight its respected daily tracking poll in order to make it come into line with the partisan breakdown that its own research tells the firm to be true. That's the same practice among most pollsters, including, it would appear, the Fox poll.
However, that doesn't mean that any individual unweighted poll can't be drastically out of line with any number of demographic realities in the country. In polling-nerd lingo, this is called the "confidence level," or the chance that any one poll accurately reflects the pool of individuals it is attempting to sample. Most scientific polls carry a 95% confidence level -- thus meaning that there's a 1 in 20 chance that any one poll is completely screwy, for any number of reasons.
Emory University political scientiest Alan Abramowitz wrote to the Huffington Post about the Fox poll, and he clearly thinks it's an unreliable outlier.
"If you believe this poll, Democratic voters are less interested in the election than Republican voters and the likely voter pool is almost evenly divided between Democratic and Republican identifiers: 41% Democratic, 39% Republican," Abramowitz said.
Gallup's own tracking of the partisan balance in the American electorate currently has Democrats enjoying a 11-point advantage over Republicans (including the "leaners" on both sides). That's in line with the conventional wisdom, which also takes into account the Democrats' edge in voter registration.
Given those data points, any random sample in a poll -- whether taken by Fox or any other news organzation -- that shows a mere 2-point edge in partisan identification for Democrats should probably be discounted to some degree.
And until more pollsters are willing to start weighting their surveys to reflect party identification, those will just be the breaks (approximately once out of every 20 polls taken). Meantime, it might help if media outlets sponsoring polls were more up front about the potential holes in their surveys. But given the amount of money poured into each poll, that might be too much to hope for.