Consider a small restaurant that's open from lunch through dinnertime. One day, every hour on the hour, a pollster counts how many people are in the restaurant. Here's what he finds:
11 a.m. - 2 people
12 p.m. - 25 people
1 p.m. - 30 people
2 p.m. - 3 people
3 p.m. - 0 people
4 p.m. - 2 people
5 p.m. - 28 people
6 p.m. - 35 people
7 p.m. - 5 people
8 p.m. - 3 people
The next day, a potential buyer comes in and wants to know how business is doing. "Great," the owners say. "We were packed for lunch and dinner - people were lined up out the door!" The potential buyer leaves, ready to put in a nice big offer for the small restaurant.
Two minutes later, in walks the restaurant's landlord, who tells the restaurateurs that she wants to double their rent. "You can't do that!" say the owners. "This place is practically empty most of the time!"
Of course, both statements are 100 percent true - and 100 percent cherry picked. If you only look at the data for the lunch and dinner rush, the restaurant is full. If you only look at the times in between lunch and dinner, the restaurant is nearly empty. Same data - different cherry-picked samples - and wildly different results.
So, what can our pollster teach us about this year's Presidential race? As the dog days of summer transition to Labor Day, you might have heard each of the following statements about the state of the Presidential race:
Clinton 10 Points Ahead of Trump Nationally
Trump Gains Ground Against Hillary, Tracking Poll Finds
Both statements are true according to different polls. And both are cherry picked when viewed in isolation.
Leading up to Election Day, you will continue to hear stories about polling on an almost daily basis. But you may not know when the media is only citing the most recent polls, or the ones that have different results than expected. You may not realize when candidates are pointing to certain favorable polls, but ignoring other ones.
As a sound consumer of polling, it's important to be aware of cherry picking. To understand and interpret this race, we need to look at the polling data in its totality. That doesn't mean we have to credit every poll, or discredit the ones we like or don't like. But we have to continue to be careful and thoughtful in interpreting the polls leading up to the election.
This post includes an excerpt from Everydata: The Misinformation Hidden in the Little Data You Consume Every Day, co-authored with Mike Gluck