It's no secret that government agencies store some of the world's most valuable data. In terms of applying analytics technology to unlock the power of what big data has to offer, government data has the potential to solve some of society's biggest challenges. And government clients are looking at big data as the next great natural resource.
The United States government, in particular, hosts some of the largest data centers in the world. A recent MeriTalk survey reported that U.S. government agencies will add a petabyte of stored data during the next two years. To put that into context, a petabyte of data is equal to 20 million four-drawer filing cabinets filled with text.
Currently, most of these organizations are just starting to explore ways to leverage analytics to manage for results, and are spending more time collecting and organizing data than analyzing it. The same study also found that 60 percent of civilian agencies and 42 percent of Department of Defense and intelligence agencies say they are just now learning about big data and how it can work for their agency.
Last week, I had the pleasure of meeting with a number of government agencies to discuss these challenges at the IBM Business Analytics in Government Forum in Washington, D.C., and they shared their experiences. These customers realize that there is tremendous opportunity to use their growing mountains of data to make better, fact-based decisions. And they feel encouraged by the White House's commitment of $200 million toward a Big Data initiative designed to better harness and utilize the massive amounts of data in the government.
The advent of Watson-like technologies such as software and hardware that instantly analyzes natural human language, and massive amounts and varieties of big data flowing from sensors, mobile devices, and the Web, can potentially help governments to find answers to questions like "which public services are most effective" and "are social benefits reaching their intended targets?" and "is energy being used efficiently in our office buildings?" These systems literally sift through the data and identify patterns and trends on the fly, then present it in a way that's easy for people to understand.
One of the greatest challenges facing government agencies is determining the most important information to look at. In the past, companies looked in the "rearview mirror," collected information from social media sites, and stored it inside a database. Then they analyzed it, which could take weeks, and brought those insights back into the business.
At IBM, we've designed a big data platform that can access, store and analyze any data regardless of how fast it is moving, what type it is, or where it resides. The platform enables clients to perform advanced analytics on data in its native form, visualize all available data for analysis, build new analytics applications, optimize workloads, and apply security and governance to big data.
Now that government can analyze any information as it happens, it can stop looking at the rearview mirror and focus on the road ahead. We're at a unique point in time where governments can better understand their citizens and the effectiveness of the services they provide.