With the election two weeks away, it's difficult to focus on anything else. In addition to daily election news, there are dozens of fresh polls - national polls, state polls, Senate-race polls. It's a flood of information and a puzzle to figure out what it all means. Nate Silver, of fivethirtyeight.com, is a master of helping to navigate what all the data means and I am addicted to his site. Last week, he wrote a blog of particular interest to me titled "Election Update: Watch New Hampshire For Signs Of A Trump Comeback."
In the post, he explained circumstances in which New Hampshire could decide the election. What I found most interesting about his blog had little to do with how New Hampshire votes but, rather, with accounting for uncertainty in predictions. Our decisions, about everything from our child's school to our company's new product, involve predictions: assumptions about how our decisions will impact the future. This means that predictions always involve uncertainty since we can only guess how the future will turn out. It is probabilistic. There are different types of uncertainty in any prediction and this is what Silver explained so elegantly.
Silver pointed out that Donald Trump trailed Hillary Clinton by almost identical margins in Pennsylvania (-7.4 points) and New Hampshire (-7.6 points) in the latest polling data. According to his model, however, Trump's chance of winning New Hampshire (16.8%) was nearly 50% higher than his chance of winning Pennsylvania (11.5%). How could that be if the margin was the same for both states?
The cause of the disparity was uncertainty. "As far as our model is concerned, the outcome in New Hampshire is more uncertain than in the other Clinton firewall states [such as Pennsylvania]." Both the quality and quantity of the polling created differences in the certainty about the polling margins. With New Hampshire, he pointed to the built-in uncertainty in polls from smaller, "more demographically idiosyncratic" states like New Hampshire, Hawaii, and Utah. ("If you don't do a good job of capturing the Mormon vote in Utah, for example, your poll or forecast is going to be way off.") New Hampshire is much more Demographically homogeneous than Pennsylvania so more subject to polling errors. Also, New Hampshire's voters swing more than Pennsylvania with national trends meaning that a change in Trump's favor in the national polls would cause New Hampshire to move more toward Trump than Pennsylvania.
All this means that the confidence in the polls in PA will be higher than those in NH, explaining the difference in the way Silver's model treats the two states despite the similar margins for Clinton.
Although Nate Silver's blog was about polling in New Hampshire, the lessons he shared about evaluating polling data apply to our own predictions about the future. In the choices we make, what are the predictive elements? What's the quality and quantity of information underlying our predictions? Once we ask these questions, we can improve our decisions in two important ways. First, we recognize the need to get better information. Second, we can modify how we rank the alternatives because of the uncertainty in some of the guesses.