What makes a public opinion poll "scientific?" If you had asked that question of a random sample of pollsters when I started my first job at a polling firm twenty-three years ago, you would have heard far more agreement than today. Now, many more pollsters are asking fundamental questions about the "best practices" of our profession, and their growing uncertainty makes it ever harder to answer the question I hear most often from readers of Pollster.com: "Can I trust this poll?"
Let's take a step back and consider the elements that most pollsters deem essential to obtaining a high quality, representative survey. The fundamental principle behind the term "scientific" is the random probability sample. The idea is to draw a sample in a way that every member of the population of interest has an equal probability of being selected (or at least, to be a little technical, a probability that is both known and greater than zero). As long as the process of selection and response is truly random and unbiased, a sample of a thousand or a few hundred will be representative within a predictable range of variation, popularly known as the "margin of error."
Pollsters disagreed with each other, even twenty or thirty years ago, about the practical steps necessary to conduct a random sample of Americans. However, at the dawn of my career, at a time when at least 93% of American households had landline telephone service, pollsters were much closer to consensus on the steps necessary to draw a random, representative sample by telephone.
Continue reading on Pollster.com