If you were watching the January 17 NBC News-YouTube Democratic Debate you probably saw a Google Trends report at some point that showed the search popularity of the three candidates.
Back on November 14, 2015, CBS News partnered with Twitter for the second democratic debate where Twitter data was used to determine the most talked about candidate that night.
Social media data is front and center in our national debates, but few understand how these share of voice charts are calculated. Twitter and Google use keywords to filter the data, but if the keywords are not inputted correctly, the share of voice will not be accurate.
Methodology Matters
In the following screenshots, I tested different spelling variations of candidate names in Google Trends.
Clinton
Sanders
O'Malley
The above tests show that Hillary Clinton's name is commonly misspelled as "Hilary Clinton." Sanders is often misspelled "Sandars" and O'Malley is occasionally spelled as "O'Mally." Misspellings like these should be accounted for when calculating share of voice.
What if a lot of people refer to Hillary Clinton as "Clinton?" Hillary's last name is arguably one of her greatest marketing assets but it's tough to measure via Google Trends. You'd have to search for the term and exclude things like "Bill" and "Foundation" to narrow in on mentions related to Hillary. Look at what would happen if we searched by last name alone. Clinton wins.
The two Democratic frontrunners have pretty established brands based on their first names. Bernie vs. Hillary? Bernie wins.
But what if we add the misspelled "Hilary?" Hillary wins.
Show Me the Data
A lot of press are using Google trends, and this is fine for a high level overview, but this probably shouldn't be used as if it were on par with poll data or professional social listening programs. Now, consider the following social media analysis compared to the Google Trends reports that have been floating all over the Internet. The following includes a few common misspellings and attempts to measure by last name without including mentions related to Bill Clinton or the Clinton Foundation.
Data: Sysomos MAP (Analyzing over 586.5 billion documents)
Date: 1/16-1/18/2016
Queries
"Hillary Clinton" OR "Hilary Clinton" OR Hillary OR [Clinton AND NOT (Bill OR Foundation)]
"Bernie Sanders" OR "Bernie Sandars" OR "Bernie Sander" OR Bernie OR Sanders
"Martin O'Malley" OR "Martin O'Malley" OR "Martin O'Mally" OR "O'Malley" OR "O'Mally"
The chart above shows a potentially much closer debate between Clinton and Sanders than if you were using Google Trends alone to show "interest."
My point here is not to convince you that Hillary Clinton is really in the lead, but maybe to make you question the reports that you see and for you to understand that the method matters when it comes to social listening. The above is not a comprehensive social listening analysis, but it should help you to understand the importance of how this information is calculated.
We Need Political Social Media Data Transparency
The stakes are too high. News data partners like Google and Twitter need to release their share of voice methodology so that people can verify the accuracy of their analysis. This is especially true if the data is being used to show the popularity of candidates and presumably how America is going to vote--we need to get this right before we enter the primaries and presidential election.
In case you need further evidence of why we should all be concerned, here's a chart from the debate on the homepage of Google Trends, which has misspelled Bernie Sanders name as "Bernie Sander."