Too Much Data, Too Little Analysis

At times, I have felt like a lonely voice crying in the wilderness, but in truth, there is a growing chorus of others who are concerned about the same thing.
This post was published on the now-closed HuffPost Contributor platform. Contributors control their own work and posted freely to our site. If you need to flag this entry as abusive, send us an email.

For many years, I have been sounding the alarm about a national security apparatus running amok, raking in more data than can make use of. At times, I have felt like a lonely voice crying in the wilderness, but in truth, there is a growing chorus of others who are concerned about the same thing.

"Too Much Information: Spy Agency Struggles To Make Sense of Data Flood" read a headline in the December 26, 2014, Wall Street Journal. The article quoted William Binney, who rose through the ranks at the National Security Agency (NSA) over a 30 year career, retiring in 2001. The NSA knows so much it cannot understand what it has, Binney said. "What they are doing is making themselves dysfunctional by taking all this data." Amen, brother.

Many recognize the problem, but constructive ways to deal with it are few. The NSA is a vast bureaucracy, and like a giant ship at sea -- it takes a long time and lots of work to get a bureaucracy to change direction. As I write, the NSA is building a new storage center in Utah that will be able to contain 100,000 times as much information as the Library of Congress. What NSA will be able to do with all that information, of course, is the big question.

To be sure, there are ideas on the table. A presidential panel recently recommended the agency shut down its bulk collection of telephone call records of all Americans (Wall Street Journal). It also recommended creation of "smart software" to sort data as it is collected, rather than accumulate vast troves of information for sorting out later. The reality is that the collection becomes an end in itself, and the sorting out never gets done. Congress is finally taking a hard look at the NSA's activities. It would appear some changes will be sought.

But the NSA is a large, powerful bureaucracy, and as such is intrinsically resistant to change, whether driven by inside or outside forces. The agency does have a serious mandate to defend the country from deadly enemies, unlike any we have ever faced before. The leadership will be tempted to view reform efforts as threats to its ability to perform its most vital functions.

But the charge of too much data, too little analysis is credible and goes to the very heart of the NSA's mission. Like other critics, I want the NSA to be more efficient and effective, not undermine its ability to function. Nor am I overly concerned about the NSA's alleged incursions into the privacy of citizens. I don't believe the people of the NSA are interested in prying into the private lives of Americans, nor that many Americans are unduly worried about it. If a certain amount of privacy must be sacrificed to ward off terrorist attacks, then that is the price we must pay.

The issue is a straightforward one of simple ability to manage data effectively in order to provide our leaders with actionable information. Too much raw data compromises that ability. That is all there is to it.

Lt. Gen. Clarence E. "Mac" McKnight, Jr., (USA-Ret) is the author of From Pigeons to Tweets: A General Who Led Dramatic Change in Military Communications, published by The History Publishing Company.

Popular in the Community