WASHINGTON -- Does the Gallup Daily tracking poll have "some methodological problems," as President Barack Obama's campaign strategist David Axelrod claimed via Twitter on Tuesday? Gallup has shown Romney doing better than most other polls, but a review of the data by The Huffington Post calls into question the theory Axelrod cited, that Gallup's polls are under-sampling black and Hispanic voters. Yet a quirk in the way Gallup asks respondents about their race may complicate comparisons of data obtained by Gallup and other polling firms' results.
On Monday, the Gallup organization began publishing daily tracking numbers on the presidential contest between Obama and former Massachusetts Gov. Mitt Romney. Each day, from now through the November election, Gallup will release results based on a rolling average of the last five days of interviewing. On Monday, Gallup gave Romney a slight advantage over Obama (47 to 45 percent). By Friday, Romney's margin had increased to three percentage points (47 to 44 percent).
But in a busy week during which 10 more survey organizations released national polls, Gallup's numbers stood out as the most favorable to Romney. Seven of the 10 show Obama ahead (by margins ranging from 4 to 9 percentage points), including the NBC News/Wall Street Journal poll released on Thursday showing Obama leading by six percentage points (49 to 43 percent).
As of this writing, the HuffPost Pollster Obama-Romney chart, based on all of the national public polls measuring the presidential race, gives Obama an edge of just over two percentage points (46.9 to 44.8 percent).
While much of the variation between the polls is owed to the statistical noise inherent in all polling, this week's Gallup results are not the first from Gallup that are more favorable to Romney than most other polls. The HuffPost Pollster snapshot below shows that Gallup's estimate of Romney's support in the surveys it has conducted has typically been 4 to 5 percentage points higher than the overall trend based on the aggregate of all polls.
The apparent skew to Romney in the Gallup results caught the attention of Alan Abramowitz, an Emory University political scientist and occasional HuffPost blogger who emailed Gallup to request the demographic breakdown of its Monday release. As reported by the National Journal's Ron Brownstein, the Gallup sample was 22 percent non-white, close to what a national exit poll showed for the midterm election of 2010 (23 percent) but less than the minority composition of the 2008 electorate (26 percent). The racial composition of Gallup's sample is "not realistic," Abramowitz concluded in an email to Gallup that he also forwarded to The Huffington Post.
The portion of a poll sample that is African-American or Hispanic is critical because of the huge differences in the preferences of white voters and of minority and non-white voters. The 2008 national exit poll showed Obama carrying two-thirds of Hispanics and 95 percent of African Americans, but losing white voters to John McCain by a 43 to 55 percent margin. In recent polls, as Brownstein reported, Obama's margins among non-whites have been generally consistent with what he received in 2008.
So if one poll includes significantly more African American or Hispanic voters than another, the poll with the smaller minority composition will be more favorable to Romney.
One limitation of Abramowitz's critique of Gallup's sample is that his basis of comparison was the 2008 exit poll. As Gallup's Frank Newport points out in a response to the Brownstein column, "It's not clear that the percentage of non-whites this Fall will grow larger than 2008." The racial composition of the 2012 electorate, Newport writes, is "still an unknown." RealClearPolitics' Sean Trende considers this argument in more detail and concludes that the racial composition of Gallup's sample "is certainly not unreasonable, and may even prove to be correct."
But predictions of what might happen in November aside, the bigger problem is the apple-to-orange comparison of the current Gallup samples to the 2008 exit poll. A far better way to determine if the racial composition of the sample is behind the Gallup "house effect" is to compare Gallup's racial composition to that of other recent polls of registered voters.
A more apples-to-apples comparison turns up very little difference on racial composition. At the request of The Huffington Post, the pollsters for the Pew Research Center, Quinnipiac University, CBS News/New York Times and Reuters/IPSOS all shared their racial composition results among registered voters. Those results appear in the table below.
These four polls have produced a fairly wide range of results over the last week, from Gallup's 2-point Romney lead on Monday to 4-point Obama advantages on the Pew Research, Quinnipiac University and Ipsos/Reuters surveys. Yet the combined percentage of Hispanic respondents and non-Hispanic black respondents was nearly identical (either 19 or 20 percent) for all but the Reuters/Ipsos poll, in which it was only three points higher (22 percent). As such, the differences in racial composition as reported do not appear to explain why the Gallup numbers have been more favorable to Romney.
It is important for poll watchers to understand that most of the the national pollsters do not weight their samples of likely voters to match demographic estimates of voters from prior elections. However, they do weight their larger samples of all adults to match U.S. Census estimates of characteristics like age, gender and race, as Frank Newport stressed in his response to the Abramowitz critique.
That weighting is critical, since most telephone surveys under-represent black and Hispanic voters in their initial, unweighted samples. They rely on the weighting to correct the bias that results from response rates that are typically lower in urban than in rural areas.
There is one remaining wrinkle for Gallup in this controversy that involves the way in which it asks about race. The U.S. Census Bureau allows for multiple responses when it asks respondents what race they are, and Gallup attempts to replicate the Census in that respect. While most pollsters ask two separate questions about race and Hispanic ancestry, Gallup goes a step further, asking five separate questions about race. They ask respondents to answer whether or not they consider themselves White; Black or African American; Asian; Native American or Alaska Native; and Native Hawaiian or Pacific Islander.
This procedure complicates the way in which Gallup weights its data and combines the responses into the single measure of race and ethnicity reported above. It also may call into question whether Gallup's racial composition numbers are fully comparable with those from other pollsters. However, the explanation of its procedures Gallup provided to the Huffington Post does reflect a good faith effort to accurately to weight its samples of adults to match U.S. Census population estimates.
None of the data reviewed in this article resolve the question of why the Gallup polls to date have been slightly more favorable to Romney than have other polls. But the data generally suggest a culprit other than the black and Hispanic composition of Gallup's samples.