WASHINGTON -- A new poll on the Massachusetts Senate race confirms what other recent surveys have shown, at least collectively. The race between Republican Sen. Scott Brown and Democratic challenger Elizabeth Warren is close and likely to stay that way for the duration of the campaign.
Seen separately, however, the same polls paint an inconsistent picture, yielding everything from a 5 percentage point Warren advantage to a 9 percentage point Brown lead. While that variation's sources are difficult to pinpoint, it is likely to persist since poll methodologies differ widely and because many Massachusetts voters have not fully engaged in the race and are still weighing their choices.
The most recent automated telephone survey, conducted on April 9 by Rasmussen Reports, shows the two candidates "running neck-and-neck," with Warren at 46 percent and Brown at 45 percent.
That very close result is similar to the current estimate produced by the HuffPost Pollster chart, based on all public polls measuring voter preferences in the Massachusetts Senate race, shown below. The chart's trend lines currently give Brown an edge of just under two percentage points (44.6 to 42.7 percent).
The relatively flat trend lines in the chart mask considerable variation among the individual polls. Specifically, in the surveys conducted this year, those results range from a nine-percentage-point lead for Brown on a Suffolk University poll in February to a five-point Warren lead on an automated poll conducted in March by the Democratic firm Public Policy Polling (PPP).
The surveys produced even wider disparities in the number of voters in the undecided category, ranging from a low of 5 percent in Rasmussen's February survey to a high of 26 percent in the Boston Globe/University of New Hampshire poll in late March.
Why the variation? Some is due to the unavoidable statistical noise that comes with interviewing a random sample rather than the full population of voters. If, hypothetically, the "true" result had been a Brown lead of 45 to 43 percent with no variation over the past several months, and if every poll had sampled 600 voters, the "margin of error" would mean that 95 percent of the polls would produce results varying from an 10-point Brown lead over Warren to a 6-point deficit. That hypothetical range of variation is roughly what we have seen in recent months in Massachusetts.
However, the methods used by the Massachusetts polls also vary widely and likely account for some of the variation in the results. For example, the Rasmussen and PPP polls use an automated, recorded voice methodology that is legally barred from dialing mobile phone numbers. The other recent surveys all claim to have sampled both landline and mobile phones, although the level of disclosure of the details of those methods also varied widely.
The omission of cell-phone only voters is potentially important, given that nearly a third of adults nationwide now live in "cell phone only" households (although the latest state level estimates by the Centers for Disease Control show a lower than average number of cell-phone-only adults in Massachusetts).
But cell phone coverage is just one aspect of the sampling. The surveys' overall sampling methods also vary. Two of the recent polls -- by PPP and MassINC -- sampled from lists of registered voters, while most of the others used "random digit dial" (RDD) techniques that sample randomly generated telephone numbers. (The Suffolk University poll has sampled from voter lists in previous polls but did not disclose the nature of their sample on their most recent Massachusetts release.) Samples from voter lists allow a more accurate selection of truly registered voters, but miss those without listed telephone numbers.
The surveys also varied in the populations they sampled. The Globe/UNH poll reported on voter preferences among all adults, while the Western New England University and Suffolk University surveys interviewed only registered voters. The Rasmussen, PPP and MassINC polls screened for voters classified as "likely" to vote, although as is typical for state-level polling, they did not disclose the portion of registered voters that they classified as likely to vote.
Differences in question order and context are another potential explanation for some of the variation. For example, the Suffolk University poll came under criticism in February for an unusual ordering of questions that may have skewed results slightly in Brown's favor.
Similarly, the bigger undecided percentage on the Globe/UNH poll may result in part from a probe of voter certainty that immediately preceded the question on vote choice. That initial question gives respondents three options, one of which is that they have "considered some candidates but are still trying to decide."
According to the full UNH report, more than a third of adult respondents (42 percent) said in February that they were still considering their choices. That probe may have encouraged some to volunteer that they are "undecided" on the vote preference question that followed.
Whatever the impact on the undecided percentage, the preliminary Globe/UNH certainty question provides a more important set of findings about the Massachusetts Senate race. While the contest has attracted attention (and campaign donations) nationwide, many Massachusetts voters are still weighing their choices. Moreover, those who have made a decision are closely divided. According to the report, those who have "definitely" decided split 51 percent for Brown to 48 percent for Warren.
Those results add up to a close race with a lot of uncertain voters, a combination likely to produce many more volatile poll results over the next six months.