We’ve made a lot of tweaks and adjustments to the HuffPost Pollster charts in the last few months. The updates have been great progress for the site -- for example, the charts are responsive on mobile now! -- but there have been a few bugs along the way, as there are with any updating process.
For a little while today our general election chart for Republican Donald Trump vs. Democrat Hillary Clinton showed a closer-than-expected margin of about 4 points separating the candidates. We published this explanation in Wednesday’s newsletter. (If you don’t get the newsletter in your inbox, go here and sign up!)
If most polls are showing Clinton up by 5 to 7 points, why is the Pollster chart only showing her up by 4? The simple answer is that the chart estimates the candidates’ trends separately — considering only the numbers for Clinton in estimating the Clinton trend line, and only the Trump numbers for the Trump trend line. The margins between the candidates are calculated from the model-produced trends. That means that the average margin between the candidates in the polls might not match the trend line margin that the model produces. Right now, that’s happening because the poll numbers are varying quite a bit for each candidate. For example, Trump’s vote support in polls that were released in the last week ranges from a low of 32 percent to a high of 45 percent. Clinton’s support ranges from 41 percent to 50 percent. So the current model estimates of Trump at 40 percent and Clinton at 44 percent make sense, even if the margin is smaller than what many of the individual polls are showing.
This is actually not different from what any model does -- you don’t calculate polling averages by averaging the margins between the candidates, but by averaging each candidate’s values. However, since our Kalman filter model is estimating a trend line, rather than a simple average, there’s room for the margins to look different from what a raw polling average would show. This is a key difference between how Pollster aggregates polls and how most others do.
The other part of the picture, which we diagnosed late this morning and have since corrected, was that a feature we added to the charts had temporarily confused the model. We added the ability to filter by respondents’ partisanship, which inadvertently caused our averages to be slightly off for a few days. The model was pulling in all of the subpopulation data, in addition to the full poll data. Today’s fix resolved that.
The partisanship filters were worth it, though. You can now create charts like the one below, which shows the polling averages for Democrats, using the “subpopulations” filter in the customization window.
What isn’t fixed is the variance in poll numbers that we mentioned above. In part, the Clinton and Trump numbers are varying because pollsters are asking the general election tossup question in different ways. If the question includes Libertarian Gary Johnson plus an other category or other candidates, Clinton’s and Trump’s numbers decrease slightly as these categories pull a bit of support from each. If the question doesn’t mention Johnson and simply has an “other” category, the Clinton and Trump numbers tend to be a bit higher.
We’re working on a fix for that, too, but it isn’t ready yet. In the coming weeks we will separate the questions that include Johnson into a separate chart. That means we’ll have two general election charts: Trump vs. Clinton with an “other” category, and Trump vs. Clinton vs. Johnson. Our default general election chart will still be the Trump vs. Clinton chart, but in response to high demand and the aforementioned methodological issues, we’re adding one that will explicitly include Johnson.
Some users have also noticed differences between the default chart and customized charts that seem to be larger than expected. That’s not a bug -- the Kalman filter model takes a long time to run, which means that it’s not feasible to use for our customization features. Because of that, once you click on “Customize this chart” and begin changing the chart, it reverts to a simpler loess regression line that can be quickly calculated and displayed. This means sometimes there will be differences between the model and the customized chart estimates. It’s also why when you switch to “less smoothing” -- which uses the loess regression -- the trend might actually look smoother than the default model.
We sincerely appreciate all of our users and the valuable feedback we get. We’re working hard to adjust to this unusual and unpredictable election season as we upgrade and continue to improve the site. We’ll publish small updates in our daily newsletter and longer articles when it’s warranted. And, of course, we don’t plan to make major changes once the fall campaigns are underway.