Sean Trende speculates that Cuccinelli supporters were uniquely shy about sharing their preference with pollsters. The Emerson College student pollsters were clever in their use of voter file data. And the BlueLabs analytics team takes a bow. This is HuffPollster for Thursday, November 7, 2013.
DID 'SHY CUCCINELLI' EFFECT INFLUENCE VIRGINIA POLLING? - Sean Trende: "If you were a moderately right-of-center voter who was fed up with Cuccinelli and told pollsters you wouldn’t vote for him, it is possible that when you walked into the voting booth, partisanship took over. Maybe these voters just skipped the race, or maybe they voted Cuccinelli. Either way, the result would be the same: McAuliffe’s share drops from what the polls projected. This would also explain why there was no accompanying drop-off for Democrats down-ballot….A related explanation would parallel the 'Shy Tory' factor, a reference to a phenomenon in British polling in the 1990s, when the Tories would routinely poll worse than their actual electoral showings. After some investigation, it was thought that voters were simply embarrassed to tell pollsters that they were voting for the Tories. Was there a 'Shy Cuccinelli' factor? Again, it’s impossible to know, but the explanation is at least consistent with what we saw both at the top of the ticket and down-ballot (people were not shy about supporting Obenshain, and really did oppose Jackson)." [RCP]
Some supportive evidence... - The theory behind the "Shy Tory" factor is that "social discomfort" of sharing their true preference with a survey interviewer leads to the understatement. If so, polls that use self-administered questionnaires rather live interviews should show less of an effect. The Virginia polls do show such a pattern. Filtering the Pollster chart to include only automated telephone or opt-in Internet polls would have given McAuliffe a lead of just 4.4 percentage points (42.5 to 38.1 percent). Filtered to include the live interviewer telephone polls put McAuliffe ahead by 8 percentages points (46.2 to 38.2 percent). [Pollster Virginia chart filtered for automated/internet polls, and for live interviewer telephone polls]
...and some contrary - A "Shy Cuccinelli" effect should have affected both the internal surveys conducted by McAuliffe campaign, by both pollster Geoff Garin and data analytics firm BlueLabs (discussed later in this newsletter). But they didn't. Both approaches used used live interviewers, but both produced results that were reportedly very close to the ultimate result. Trende continues: "What we’re really hypothesizing is that McAuliffe’s support among moderately center-right voters was soft due to mutual dislike of both candidates. Northam’s support was firm since he didn’t cut a strong profile, and Obenshain’s was firm since he was perceived as a moderately center-right candidate...this could explain Geoff Garin’s internal polling for McAuliffe: With more sophisticated survey techniques than public polls use, it might have been able to better identify weaknesses in McAuliffe’s support." The problem with that line of reasoning is that the differences in methods used by the campaign pollsters mostly involve their sampling and post-survey weighting. They ask the same vote preference questions and use the same kinds of interviewers as public polls, so it's not clear how their techniques would have reduced a "Shy Cuccinelli" effect.
Remember Lenski's Law - HuffPollster urges those scratching their heads over the puzzling Virginia polling to remember what we like to call Lenski's Law of Polling Misfires. It is named after exit pollster Joe Lenski, who earlier this year reminded us that investigations of some of the most infamous polling failures of the last two decades -- including exit poll problems in Florida in 2000 and nationwide in 2004, and errors in surveys conducted before the New Hampshire primary in 2008 and Gallup's troubles in 2012 -- found not one big culprit, but a series of small errors all creating statistical bias in the same direction. **"Every survey has errors," Lenski told us, referring to the many choices that pollsters make about how they design their surveys that often create small random errors. "It's just a matter of, are they small and do they cancel each other out...When they're small and they're all in the wrong direction, they make you look bad**." In other words, several little factors -- rather than one big one -- may explain the understatement of Cuccinelli's vote in the polling averages. [HuffPost]
VIRGINIA'S CLOSE FINISH AND THE STUDENTS WHO 'SAW IT COMING' - Don Frederick: "A check of the polls taken over the contest’s last two weeks...shows that the one outfit coming closest to nailing McAuliffe’s margin is based in Boston. And run by students. That would be the Emerson College Polling Society, which in its Oct. 25--30 survey showed McAuliffe with a 2-percentage-point edge, a result that at the time seemed an outlier...'We’ve had a pretty good track record,' the Polling Society’s president, 21-year-old Siobhan Robinson, told us last night. And with the outome in Virginia, 'It’s really great to see our work pay off.' Robinson, a senior from Miami, said that as the group’s efforts have gained increasing attention from political journalists, some have inquired 'if there are any adults involved.' There’s a faculty advisor assigned to the society, but she and a crew of five to eight other students devise their poll questionnaires and crunch the numbers, Robinson said." [Bloomberg]
Faculty advisor explains methods - Emerson College Prof. Spencer Kimball, the advisor to the Polling Society, took to the listserv of the American Association for Public Opinion Research (AAPOR) on Thursday to comment on the performance of Virginia's polls (we share his comments with permission): "One of the major factors we saw in forecasting the race was estimating the voter registration of the electorate. In VA you only register as a voter and not by party which led to varied parameters. Also, Robert Sarvis who was a third party candidate, was near 10% in many polls including Emerson which had him at 12.5%, we were expecting to see that number drop to under 10% which it did at 6.5%. However, while the conventional wisdom was that Sarvis, the libertarian, was hurting the Republican, the Washington Post poll on Oct 27 had asked if Sarvis was out of the race who would you vote for- with the results showing a split between the candidates. Another difference between the Emerson poll and the other polls was the weighing by congressional districts and not counties or regions."
We asked Kimball to clarify what he meant by "estimating" voter registration. His response via email: "We estimated the voter registration based on primary vote history and parameters set by other surveys and self reports measures we found after the 2012 election, which led us to a slight democrat lead of about 3 points. The next question was how large of a turnout was going to come from the Democrat and Republican voters v. Independents. Here we based our forecast on exit polls and voter turnout figures from 2009 that Independents and Republicans would account for just under a third of the vote and the Dems would account for about 36%."
MCAULIFFE POLLSTER SAYS OBAMACARE STANCE HURT CUCCINELLI - Greg Sargent interviews McAuliffe pollster Geoff Garin: "The McAuliffe campaign tested Cuccinelli’s position and rhetoric on the health law and found it alienated more voters than it attracted...Garin said the campaign specifically polled on Cuccinelli’s bragging point as the first attorney general to target the health law, and that it was a loser for him. 'We tested Cuccinelli’s brag that he was the first attorney general to sue to stop Obamacare,' Garin said. 'That actually made more voters less likely to support him than more.'...[He ads] that even among some of those who disapproved of the law, Cuccinelli’s position supported the Dem narrative that Cuccinelli is too extreme. 'A majority disapproved of the Affordable Care Act, but in Virginia, as elsewhere, we found that a lot of these voters want to fix the law,' Garin said. 'Cuccinellis’ position on Obamacare actually supported what we were saying about him, which is that he was extreme and supported a national Tea Party agenda.'" [WaPost's Plum Line]
What about the shutdown "bump?" - Yesterday's HuffPollster included a portion of a post-election email in which Garin said that McAuliffe's lead was always in the 2 to 4 point range, "with the exception of the a bump up during the shutdown." Some readers felt that sounded like a big exception, so we asked Garin to clarify. Here is his response: "Terry’s lead in the 3-way peaked at only 6 points on Oct 22. The 8% undecided in that poll were Romney voters by 16-points, so the ultimate split among them accounts for some of the narrowing back to 3. And in statistical terms the difference between a 6 point spread in a poll compared to a 3 point spread in a poll is not all that meaningful." [HuffPollster]
The list vs. random digits debate continues - In yesterday's HuffPollster, Garin attributed the accuracy of his polling to samples drawn from lists of registered voters and the "combination of vote history and modeled party scores" available on those lists. Garin's comments drew a strong reaction from Republican pollster Alex Lundry on Twitter: "Maybe this will convince all those GOP pollsters that inexplicably STILL refuse to sample from the voter list...Completely baffled by the reluctance on the right to use listed samples - we know SO MUCH more about respondents and non-respondents!...If GOP sincerely wants to run data-driven integrated campaigns, then good place to start is knowing something about the voters they survey." [@AlexLundry via Storify]
THE DATA FIRM BEHIND MCAULIFFE'S WIN - Ben Pershing: "McAuliffe also deployed numbers in far more sophisticated fashion [than Cuccinelli], and he did it with the help of many of the same wonks who powered President Obama to victory in 2008 and 2012. Like Obama, McAuliffe leaned heavily on data, compiling reams of information on voters across the commonwealth. And that hefty bank account gave him the resources to put it to good use. The bulk of McAuliffe’s data work was done by BlueLabs, a firm launched earlier this year by veterans of the Obama analytics operation. The company has roughly 20 employees, and nearly all of them worked for the president’s campaign last year….Although the final vote margin was closer than public polls suggested, the model BlueLabs built turned out to be accurate, according to the McAuliffe campaign….In addition to advising McAuliffe from the outside, Blue Labs also had a staff member, Harrison Kreisberg, embedded with the McAuliffe campaign to weigh in during strategy sessions. The McAuliffe campaign and the Democratic Party of Virginia paid BlueLabs just over $200,000 combined for the firm’s services, according the Virginia Public Access Project." [WaPost]
A further look at how the campaign used data - Alexander Burns: "In February, McAuliffe took a large-sample poll of 10,000 Virginia voters. That huge tranche of data allowed the campaign to create a detailed model of the off-year electorate…. That model – updated repeatedly over the course of 2013 – did more than just give McAuliffe a clear-eyed view of the state in which he was competing. The information included in that model, which also encompassed data on voter participation collected in previous campaigns, guided granular decisions about the McAuliffe ground game…The McAuliffe campaign offered up numbers from several targeted areas to prove the point. Prior to the election, the McAuliffe modeling predicted that without extensive campaign efforts, McAuliffe would win 53.6 percent in the Norfolk media market, 50.1 percent in the Richmond market and 55.3 percent in the Washington D.C. media market. The model predicted that with a heavy-duty campaign operation, including field, mail and digital – McAuliffe would slightly increase his vote share to 55.1 percent in Norfolk, 52 percent in Richmond and 56.6 percent in Washington. The final score closely mirrored those projections, as McAuliffe took 55.4 percent in Norfolk, 51.7 percent in Richmond and 56.7 percent in the D.C. market." [Politico]
HUFFPOLLSTER VIA EMAIL! - You can receive this daily update every weekday via email! Just enter your email address in the box on the upper right corner of this page, and click "sign up." That's all there is to it (and you can unsubscribe anytime).
THURSDAY'S 'OUTLIERS' - Links to more news at the intersection of polling, politics and political data:
-More Americans oppose a ban on trans fats than favor it. [Pew Research]
-More Americans tuned into politics in October despite their negative feelings about their political leaders. [AP, topline]
-Americans distrust "government provided health care"...unless it's Medicare. [YouGov]
-Ari Shapiro argues President Obama shouldn't worry about his sinking poll numbers. [NPR]
-PPP (D) finds mixed numbers for Ted Cruz. [PPP]
-Ruy Teixeira says Virginia is tipping from purple to blue. [New Republic]
-Nate Cohn says the 2013 results do not bode well for Democrats. [New Republic]
-Robert Sarvis didn't cost Ken Cuccinelli, reports Chris Cillizza. [WaPost's The Fix]
-Brent Benson finds evidence that endorsements from Boston mayoral candidates who lost in the primary moved votes in the general election. [Mass Numbers]
-Americans remain basically supportive of the U.S.'s role in improving health conditions in developing countries. [Kaiser]
-ESPN conducts an unscientific poll on the motivational effects of free bacon. [ESPN]