Comey Letter Swung Election For Trump, Consumer Survey Suggests

Donald Trump’s election—and now, his presidency—have invited fundamental questions about the trustworthiness of the news media, of science and measurement, and, indeed, the very idea of objective truth. When President Trump levels his charge of “fake news” at mainstream news organizations, Exhibit A is Trump’s victory in the election despite widely reported polling and statistical models that suggested he would lose.

As my market research industry colleague, Josh Chasin of Comscore said in a recent commentary:

“The fact that the polls all said he’d lose, but then he won, serve to prove to Trump and those around him that mainstream media is fake news, while nonsense like Fox or Breitbart or Infowars are legit. And let’s face it― our collective worlds were rocked by this twist, and even those of us steeped in logic and leaning left have been doubting the veracity of things we have taken to be true our whole lives.”

Many Americans—and particularly those of us working in data-driven businesses—would like to see a credible, fact-based explanation for why the polls seemed to indicate a Clinton victory, but the election instead produced President Trump. Before offering such an explanation, allow me to summarize what the polls were actually saying.

The false predictions of a comfortable Clinton victory were based on state and national polling throughout the campaign that showed her with a fairly persistent lead, as documented by the website FiveThirtyEight, among others. Even in the final week of the campaign, Clinton’s lead remained between 3 and 5 points, finishing at 4 points on Election Day. Of course, Clinton won the national popular vote by 2 points, even as she lost the Electoral College vote due to very narrow defeats in Michigan, Pennsylvania, and Wisconsin.

My purpose is not to suggest there was an alternate way to confidently predict the election because the Electoral College outcome hinged on merely 80,000 votes in the three aforementioned states. But I do believe it was possible to show that the possibility of a Trump victory was rising more rapidly in the final week than opinion polls—and related prediction models—showed.

My firm has been tracking face-to-face consumer conversations about brands for more than a decade. Our methodology is to conduct a daily online survey among consumers to ask them about the product categories and brands they have been talking about in the last 24 hours. Thus we are using a survey to measure behavior rather than opinion data. Third party statistical modelers have validated this methodology by showing that when combined with social media data it can predict between 5% and 25% of a brand’s sales, depending on the category.

Although it is not our main line of business, every four years since 2008, we have added a few special questions to pick up the daily conversation about presidential candidates during the General Election campaign. Only after Election Day last year did we go back to see what the data showed, and it was startling.

The first thing to know is that people were talking very negatively about both Trump and Clinton, in contrast to the mostly positive conversations we see for products and brands. Between Labor Day and Election Day, 53% of all the Trump conversations were negative about him, and 20% were “mixed” positive and negative, while only 26% were purely positive. If you subtract the negative and mixed from the positive, you get a “net sentiment” of -47. For Clinton, it was 43% negative and 21% “mixed” compared to 33% positive, producing a “net sentiment” of -30. That both candidates were so deeply into negative territory tells us something about politics generally, and also about these two candidates, who performed worse than Obama, Romney, and McCain in the last two elections.

More important than the absolute results for each candidate were their relative performances, as well as the trends over time. While both candidates were always firmly in negative territory, Clinton nevertheless enjoyed a persistent lead over Trump that opened up after the first debate. Both candidates experienced significant drops in the immediate aftermath of the infamous audio recording of Billy Bush and Donald Trump, although Clinton still had the advantage.

Most decisively, there was a sudden change in the net sentiment results that followed immediately after FBI Director James Comey released his Oct. 28 letter to Congress about a renewed investigation of Clinton emails. Immediately afterwards, there was a 17-point drop in net sentiment for Clinton, and an 11-point rise for Trump, enough for the two candidates to switch places in the rankings, with Clinton in more negative territory than Trump. At a time when opinion polling showed perhaps a 2-point decline in the margin for Clinton, this conversation data suggests a 28-point change in the word of mouth “standings.” The change in word of mouth favorability metric was stunning, and much greater than the traditional opinion polling revealed.

Based on this finding, it is our conclusion that the Comey letter, 11 days before the election, was the precipitating event behind Clinton’s loss, despite the letter being effectively retracted less than a week later. In such a close election, there may have been dozens of factors whose absence would have reversed the outcome, such as the influence campaign of the Russian government as detailed by US intelligence services. But the sudden change in the political conversation after the Comey letter suggest it was the single, most indispensable factor in the surprise election result.

This conclusion helps us to understand how it is possible that the polls were generally correct about a Clinton lead through most of the campaign, but nevertheless Trump still won because of a late October surprise. In other words, pollsters and the media were likely correct that Clinton was “winning” during most of the campaign.

Our conversation data also raise a different, but fundamental, question about the predictive power of public opinion data in predicting behavior, at least given the unique features of a campaign that featured two unpopular candidates, non-traditional campaigning, and a late October surprise.

My colleagues and I have developed several theories for why our conversation data showed so much more movement than the traditional opinion polling.

1. Behavior predicts behavior better than attitudes and opinions do, even when the behavior is measured by a survey. Presidential elections involve not only a candidate preference choice, but also a choice to vote or stay home. While the Comey letter did little to alter the measured preference for Clinton versus Trump, it did make a difference in voter conversations, and also in the motivation of Democrats to vote. The drop in net sentiment for Clinton was largest for Democrats at -19 points, while it remained unchanged for Republicans. Meantime, in the week of the Comey letter release, Trump’s net sentiment improved by 21 points among Republicans and by 6 points among Democrats. Thus it appears that the experience of these conversations depressed Democratic turnout at the last minute, while increasing it for Republicans, making Trump’s narrow victories in states like Wisconsin, Michigan, and Pennsylvania possible.

2. The Invisible Offline Conversation Matters. There’s a lot of focus today on the conversations and news sharing that happen on Twitter, Facebook, and other social media platforms—but we need to think of social media as just the tip of an iceberg. In addition, there is a large offline conversation that is hidden beneath the surface, difficult to see and measure. That conversation may or may not resemble what we see online. That is why we have our offline conversation surveys, and a measurement platform that also includes online social media data.

3. Humans are a herding species, susceptible to sudden changes in direction when confronted with the right stimuli, and when surrounded by other people of like-mind who are impacted by the same stimuli. Comey’s letter provided the stimuli for a sudden change in the peer influence dynamic that drove the election outcome.

Political consultants and commercial marketers alike have relied on a model that presumes voters and consumers act according to rational, individual choices that they can express and explain. What we are learning is that emotion and peer influence play much bigger roles in influencing behavior than previously understood. For this reason, we have to assume that opinion polls are just one important indicator, and we should look for other potential indicators as well.

Even the best science will draw a wrong conclusion from time to time. That doesn’t make it fake. Rather, it means we should keep seeking out new tools and techniques that will improve our rate of success.

Note: This post represents an adaptation of a January 2017 post by the same author on his company’s website in January 2017. The earlier post was written for a marketing research audience.