A new round of national polls is telling a consistent story: Obama's overall ratings remain stable despite rising concerns about the scandal stories gripping Washington. A commendable effort to score pollster disclosure fails to disclose the pollster scores. And one pollster got it right in British Columbia. This is the HuffPost Pollster update for Tuesday, May 21, 2013.
NEW POLLS TELL SAME STORY- As summarized by NBC's First Read: "[W]e now have the results from three different polls (CNN, Pew, WaPo/ABC) released in the last 36 hours. And they provide a consensus on three different stories. One, the public believes there was wrongdoing by the IRS and in the aftermath of the Benghazi attack. Two, this belief of wrongdoing hasn't affected President Obama's standing." [NBC]
Obama's job approval unchanged - The polls that tracked Obama's job approval rating -- the CNN/ORC and ABC/Washington Post surveys, plus another national poll released on Tuesday by the American Research Group and the ongoing daily tracking surveys from Gallup and Rasmussen Reports, show virtually no net change in Obama's job rating since April. An average of the five polls rounds to 50 percent in April and 50 percent last week. [Pollster chart]
IRS and Benghazi controversies are potentially damaging - Yet the recent controversies roiling Washington present the "potential for significant damage" to the Obama administration, according to the Post/ABC poll: "Americans by a vast 74-20 percent see the IRS' behavior as inappropriate, with most feeling that way strongly -- and 56 percent see it as a deliberate attempt to harass conservative organizations, not a mere administrative error. The public divides on whether or not the administration is honestly disclosing what it knows about the IRS' actions; 45 percent suspect a cover-up, 42 percent instead see full transparency...Further, on the attack on the U.S. diplomatic mission in Benghazi, Libya, last fall, suspicions of a cover-up rise to a majority, 55 percent. And in this case only a third of Americans are persuaded that the Obama administration is disclosing honestly what it knows about what occurred." [ABC News]
But economy boosts Obama - Yet Obama's job rating remains stable. Why? One reason is the improving economy. Washington Post: "After two months of clearly negative ratings over his handling of the economy, Obama has climbed back to about even, with as many now approving as disapproving of his performance on this front. The president also holds a nine-percentage-point advantage over congressional Republicans on the issue." [WaPost]
Wealthy most optimistic - WaPost's Jim Tankersley: Fifty-two percent of white Americans earning $50,000 a year or more are optimistic about the national economy, a 13-percentage-point increase from December...Thanks largely to that shift and to persistent optimism among higher- income nonwhites, economic optimism among all Americans is at its highest level since early 2009. The change tracks the performance of the Standard & Poor's 500-stock index, which has risen 20 percent in the past six months. Higher-income Democrats and, perhaps most notably, Republicans are all feeling the effects. [WaPost]
Another poll confirms growing confidence - Gallup's Alyssa Brown: "Americans were more confident in the economy last week than in any week since Gallup began tracking economic confidence daily in 2008." [Gallup]
MEASURING POLLSTER TRANSPARENCY IN 2012 - The annual conference of the American Association for Public Opinion Research included one particular presentation that we followed with great interest. Researchers at the Marist Institute for Public Opinion presented results from an extensive effort to measure how much pollsters who tracked the presidential race in 2012 disclosed basic facts about their methodologies. They collected all of the publicly released polls, national and statewide, and scored the quality of disclosure by 189 different survey organizations and media partnerships. They checked the most recent release by each organization for disclosure of 11 specific items mandated by AAPOR's Code of Ethics and a wider list of 22 items proposed for inclusion by AAPOR's still-under-development Transparency Initiative. [AAPOR's Code of Ethics, Transparency Initiative, TI Elements]
Longtime readers may remember that we called for just such an effort in 2009: We proposed to "gather and record whatever methodological information pollsters choose to publicly release" and then "score the quality of disclosure of every public survey release. Aggregation of these scores would allow us to rate the quality of disclosure for each organization and publish the scores alongside polling results." Such an effort, we wrote, "could create a set of incentives for pollsters to improve the quality of their disclosure in a way that enhances their reputations rather than making them feel as if they are handing a club to their enemies." [Pollster.com]
Easier said than done - We learned the hard way, however, that scoring disclosure is remarkably difficult, because as Marist's Senior Analyst Natalie Jackson explained at the AAPOR conference on Saturday, "defining these things is a very difficult task." Scoring many of the mandated disclosure items requires subjective judgement or expertise in survey concepts (or both). "We had meetings that were several hours long trying to determine how to decide these things, how to put it into a research design, how to operationalize it," Jackson explained. "There's a lot of room for interpretation." Even then, Marist had to use 20 student interviewers as coders, assigning each of 189 releases to two different students for separate coding, with the primary authors stepping in to resolve differences.
'Nobody's Perfect' - The results: Only 9 percent of the organizations disclosed all of AAPOR's 11 minimum disclosure items (who sponsored the survey, who conducted it, survey dates, sample size, margin of error, the population surveyed, the mode of interview, a description of the sample design, information on weighting, full question wording and contact information), although 83 percent disclosed at least half. Not one of 189 organizations reported all of the 22 items the Transparency Initiative proposes to mandate, though 50 percent disclosed at least half. But Jackson also reported "some really good trends." AAPOR members, academic surveys and organizations that have endorsed the Transparency Initiative all scored significantly higher.
A Key Omission - Marist's initiative is commendable, and it significantly advances the move toward greater transparency by pollsters. But an ironic omission illustrates how much work remains: Marist has so far opted against reporting the scores of individual survey organizations. "We didn't want to come to AAPOR and point fingers, and let that detract from the overall issue of transparency, " Jackson explained. Since "we're part of our own sample," she added, Marist risked the perception that they were only interested in "pointing out your competitors' scores." So they chose to report only aggregated scores.
We hope the Marist researchers reconsider. They have done the hardest work of all, in working through rules for the coding. Publishing the scores would likely upset those who scored lower, but it would also create an incentive for those organizations to change their practices, especially if Marist undertakes another round of transparency coding in the future.
LIBERAL PARTY POLLSTERS GOT IT RIGHT IN BRITISH COLUMBIA - The Province's Michael Smyth: "For the first time, The Province has obtained polling data compiled exclusively for the Liberal party — numbers that showed the NDP bleeding late-campaign support to the Liberals, especially among older voters and women. The Liberals' internal polling was done by Praxicus Research, a company owned by Dimitri Pantazopoulos, a Conservative pollster who worked for Prime Minister Stephen Harper and Toronto Mayor Rob Ford among others....The Liberals then did something other pollsters didn't: they "weighted" the results by age group, according to their likelihood to vote. For example: even though the NDP was leading among eligible voters aged 18-34, research shows younger voters are less likely to actually cast a ballot than older voters." [The Province]
TUESDAY'S 'OUTLIERS' - Links to more news at the intersection of polling, politics and political data:
-Charlie Cook argues that GOP hatred of Obama blinds them to public disinterest in the recent scandals. [National Journal]
-Seth Masket reviews two theories for why the scandals haven't been hurting Obama's approval ratings. [Pacific Standard]
-Brendan Nyhan is unsurprised by the shift from scandal coverage to tornado coverage. [Twitter, PDF]
-Russia's only independent polling agency says it may close. [NYTimes]
-Despite a small dip, Hillary Clinton "remains among the most popular secretaries of state in recent history." [WaPost]
-Acceptance of gay and lesbian "relations" jumps to 59 percent. [Gallup]
-The Pew Internet & American Life Project releases a new report on teens, social media and privacy. [Pew Research]
-Nate Cohn responds to Ross Douthat, finds no evidence that marriage or religion suppports a surge in suicides. [TNR]
-Video published from the MPSA panel featuring John Sides, Larry Bartels, Nate Silver, Ezra Klein, Simon Jackman, Lynn Vavreck, and Drew Linzer. [Monkey Cage]
-Matthew Jason remembers a mentor and friend, Public Opinion Strategies partner Stephen Kinney. [POS]
CORRECTION:The original version of this article misstated the numbers of items proposed for inclusion by AAPOR's Transparency Initiative as 200. The correct number is 22.