An exhaustive 2008 academic study of polling in the years after 9/11 can help make sense of what might be seen as confused public opinion today on privacy and government surveillance. It also suggests concerns about surveillance may be underreported as some people particularly sensitive about privacy might not take surveys.
Pollsters jumped into action after bombshell disclosures early this month about vast National Security Agency collection of telephone and Internet records. Polling observers quickly noted widely varying results, some indicating Americans were sanguine about the surveillance and others signaling greater alarm.
Chris Bowers of the Daily Kos went so far as to say "you should ignore polls on the NSA leak" because "it appears beyond anyone's reach to accurately measure what millions of people actually think about those surveillance programs."
Others instead offered guidance in interpreting the results -- like HuffPost Pollster Mark Blumenthal, here and here (and in 2006, during an earlier NSA surveillance controversy), Democratic pollster Mark Mellman and Gallup's Frank Newport.
All these commentators noted that question wording can make a big difference. That's always true in polling but especially so on a topic like privacy vs. surveillance, when the broader context includes fear of terrorism, and when many people may not be paying very close attention to the issue.
In fact, Mellman noted that those paying the closest attention -- the 27 percent "very closely" following news stories about the government collecting phone records -- were most critical of the NSA activity. That finding comes from a Pew Research Center/Washington Post poll, and Jon Cohen of the Post's independent polling group Capital Insight provided me these crosstabs, illustrating the point starkly:
With caveats in mind, a critical reading of the poll results can yield coherent and important insights. Stability or shifts in opinion (overall and among subgroups, like partisans) can be monitored when comparable questions are repeated over time. Either way, it's an issue that won't go away in the age of Big Data.
Many polls asked about government surveillance as Americans reacted to Sept. 11, 2001, terrorist attacks. Amy Corning and Eleanor Singer -- a preeminent researcher on privacy and confidentiality in public opinion research -- studied questions asked in more than 100 surveys in the five years after 9/11 and published patterns they found in a 2008 report for the National Research Council of the National Academy of Sciences.
In one finding with particular relevance today, Corning and Singer noted attitudes about privacy are not well-formed, and thus "(t)he framing of survey questions, in terms of both wording and context, strongly influences the opinions elicited."
Also: "Attitudes toward surveillance and the appropriate balance between rights and security are extremely sensitive to situational influences, particularly perceptions of threat." They quoted a 2004 article in the American Journal of Political Science by Darren W. Davis and Brian D. Silver: "(W)hen they feel threatened, people who previously protected civil liberties and personal freedom may compromise on these values for greater security."
Among other conclusions by Corning and Singer, both of the University of Michigan's Institute for Social Research:
- Americans in the years after 9/11 were "quite willing to endorse specific (surveillance) measures, especially when the measures are justified as necessary to prevent terrorism" and "when it is aimed at specific racial or ethnic groups, when it concerns activities they do not engage in, or when they are not focusing on its potential personal impact." Also, few felt government anti-terrorism efforts affected their own privacy.
- However, "As time from a direct terrorist attack on U.S. soil increases, the public is growing less certain of the need to sacrifice civil liberties for terrorism prevention, less willing to make such sacrifices, and more concerned that government counter-terrorism efforts will erode privacy." (2013 question: What effect might the Boston Marathon bombings, less than two months before the NSA disclosures, have had on public opinion on these issues?)
- Support for monitoring private communications, especially telephone and email, was consistently lower than for surveillance in public places and tougher security for travel and building access.
The authors also addressed the possibility that the polling was affected by "nonresponse bias" -- whether people who took the surveys held different views than those who refused to cooperate.
Earlier research by Singer and Stanley Presser of the University of Maryland, in a chapter for the 2006 book Advances in Telephone Survey Methodology (sub./pay req'd.), found that concerns about privacy and confidentiality helped reduce cooperation with government surveys including the Census. The authors found that generally "the net effects of such attitudes, although statistically significant, are relatively small." But they also wrote: "If survey measures are strongly related to attitudes about privacy, confidentiality, or burden, then nonresponse bias is likely."
Poll questions about NSA surveillance certainly would be strongly related to attitudes about privacy.
Corning and Singer addressed this issue in their 2008 work, as well. They noted they had no way to detect or estimate nonresponse bias, nor to rule it out. As for which direction any bias might run, the authors noted that fewer than 1 in 10 of the polls they studied focused largely on privacy or civil liberties, so there seemed little chance of bias from overrepresentation of people drawn to those topics. Corning and Singer concluded it's more likely bias would run the other way, if some people don't take polls for perceived privacy reasons: "We expect that whatever bias exists will be in the direction of excluding those most concerned about privacy and that the findings reported will tend to underestimate levels of privacy concern."
Such bias, if it exists today, may be compounded by the finding in the graphic above from the Pew/Post poll, showing a relationship between paying close attention to the NSA story and viewing the tracking program as unacceptable. If there's similarly a correlation between attention to privacy issues and refusal to take polls because of concern about one's own privacy, that could exacerbate underreporting of privacy concerns and surveillance criticism in the poll results.
Proving that empirically, though, would be quite difficult and expensive, as is any research on why some people refuse to take surveys.