The results of the instant snap polls by CBS News, CNN/ORC and the Democratic pollsters at Democracy Corps all show overwhelmingly positive responses to President Barack Obama's State of the Union address from Tuesday night. Yet if past history is a guide, these impressions will not translate into a "bump" -- a lasting, measurable change in public opinion. These snap polls, for reasons that have never been clear, almost always yield an immediately favorable response.
One of the big challenges pollsters face in measuring reactions to the State of the Union address is that reaching a fresh random sample of adults within minutes of the speech is nearly impossible. Moreover, not everyone watches the speech. So those that try to measure reactions to the speech compromise, and aim to interview only those who say they have watched the speech.
In order to get a large number of interviews completed quickly -- particularly when the speech finishes after 10 p.m. on the East Coast -- pollsters also need to recruit their respondents ahead of time. Like CNN, they use standard national telephone surveys conducted a week or or more in advance, ask respondents if they plan to watch the speech, and then call back those who say "yes" within minutes after the speech has concluded. Or like CBS News, they use a similar procedure to interview and reinterview those who plan to watch the speech sampled from the online Knowledge Networks panel (more on that methodology here).
But the bottom line is, the slice of Americans who say they watch the speech almost always skews heavily toward fans of the president. In both the CNN and CBS polls last night, Democrats outnumbered Republicans by wide margins and, as the table below shows, that pattern is typical. The same partisan skew was present for Obama's previous addresses to Congress and for President Clinton, and the reverse occurred for the addresses of President George W. Bush: Republicans typically outnumbered Democrats by roughly the same margins.
One possible surprise of last night is that the presence of new Republican House Speaker John Boehner did not result in a more balanced partisan audience composition. The partisan composition was considerably closer in 1995 when Newt Gingrich assumed the speaker's chair while Bill Clinton was president, and in 2007, when Nancy Pelosi took it back for the Democrats and appeared for the first time seated behind President Bush.
The partisan skew alone does not explain the overwhelmingly positive response last night, but that too is typical. Consider the first question asked by CNN about the overall reaction to President Obama's speech. Virtually all of their respondents (84 percent) reacted positively, either very (52 percent) or somewhat (32 percent), and only 15 percent had a negative reaction. However, as the table below shows, that response matched the usual pattern measured by CNN for every State of the Union address since 1998. In fact, Obama's marks last night exactly matched their overall average of 84 percent positive, 15 percent negative.
CNN also asked a before-and-after question that showed the speech moved many viewers to a more positive impression. Before the speech, 61 percent agreed that Obama's policies "will move the country in the right direction." After the speech, the number increased to 77 percent, for a gain of 16 points. Yet, as the table below shows, State of the Union speeches always move their audiences to more positive impressions. Once again, the opinion shift during the Obama speech matches the overall average almost exactly.
The CBS News/Knowledge Networks surveys have less of a history but show a very similar pattern. Last night, 92 percent of their speech viewers expressed approval for the "proposals the President made in his speech," a slightly better result than last year (83 percent) but about the same as for Obama's 2010 address (91 percent). Reactions to Obama's first three addresses as measured by CBS are slightly better than reactions to the last three by George W. Bush.
CBS also reported that approval for Obama's "plans for the economy" jumped from 54 percent of respondents before the speech to 81 percent after. Last year's address involved a similar jump, from 55 percent approval to 76 percent.
Yet as our regular readers are probably tired of hearing by now, these uniformly-positive reactions among viewers almost never translate into a sustained, measurable change in perceptions or ratings of the president in subsequent representative samplings of the general public. As Gallup reported Tuesday, they have measured since 1978 "only four instances when a president's approval rating increased by at least four percentage points after a State of the Union address, with Bill Clinton the president in three of these instances and George W. Bush in the other."
Why the conflict? Perhaps the advance notice primes respondents to watch the speech more attentively than the rest of us. Perhaps their reactions are comparable to what other Americans experience, but the number of positively affected viewers is simply too small to make a measurable dent in the overall ratings. Perhaps the impression that the speech makes fades almost immediately in the face of the news coverage, commentary and other news events that follow.
We may never know for sure, but the bottom line is that it will take a week or two and another round of representative national surveys before we learn whether this speech has coincided with a meaningful a change in perceptions of the president. Unfortunately, the overwhelmingly positive instant-response polls do not tell us much.