This post was originally published on Cisco's Technology News site.
Thanks to the ability to quantify and qualify personal data and metadata through the plethora of social networks, apps, and device usage among the masses, large companies are able to target customers and reach audiences on a more granular level than ever before, making big data a significant commodity in the commercial market. Now there is proof that big data is becoming even more widespread and having an impact on the world from a socially conscious perspective. Today, with the help of humanitarian data scientists, mission-driven organizations are aiding people living in extreme poverty and enduring the food crises.
Max Richman, a lead data scientist at Mobile Accord, volunteers for DataKind, a community of data scientists and activists working together to spark social action. He says businesses are ahead of the game as far as harnessing big data, but non-governmental organizations are starting to catch up and get more creative.
"In a certain sense, commercial partners get the possible utilization of big data and want to use it more quickly," shared Richman. "But we're seeing interest across the broad social sector with groups wanting to help people through mobile phones and other tools."
Richman is currently working with Geopoll and the World Food Programme (WFP) on a project that revolves around using mobile text to conduct food security surveys in conflict regions. Since the initiative started, collecting data through SMS to assess the availability of food in households in the Democratic Republic of the Congo has proven to be more timely and cost-effective than previous surveying methods. Richman says using mobile phones has decreased survey costs from $40 to $5. In addition to implementing the mobile surveying tactic, Geopoll built and interactive dashboard to help WFP see how numbers are trending.
How valuable is big data on its own?
Fellow DataKind volunteer Kush Varshney is a researcher at IBM. He says the convergence of activism and big data is creating great opportunities for NGOs, but it's going to take more than collecting data to generate tangible results.
"Data isn't valuable without analysis or converting insights and information," said Varshney. "So just having raw data is great, but until you do something with it or until you have some hypotheses it's not going to useful."
Varshney recently wrapped a DataKind project with an organization called GiveDirectly, which makes unconditional cash donations to impoverished households in Africa via mobile telephony. Using satellite images to identify the poorest villages in Kenya and Uganda, Varshney and his team were able to help GiveDirectly cut costs and time spent typically spent when data is collected manually.
"From Western Kenya to Uganda, we identified the types of roofs homes had to determine a village's relative wealth," explained Varshney. "There are two types of roofs. Iron is one and thatch, which is made of grass, is the other. Thatch roofs are common in the poorest areas."
Varshney said working with satellite images was beneficial for more than one reason.
"A lot of times, getting information on the ground is difficult because of local politics," he shared. "So if it's possible, collecting data through satellites is a good idea."
Another way data is being aggregated for social impact is through internet scraping. In March of 2013, Richman and a team of data ambassadors joined forces with the World Bank to examine the food crisis and its correlation to extreme inflation. They did this by scraping several websites and creating new datasets around food pricing.
"We had volunteers at one of our hackathons help us," said Richman. "Anyone who knows how to use Google can do internet scraping."
Richman's team used a number of different tools including Python, ScraperWiki, and the Wayback machine on the project. The goal was to see if the data scientists could mine free resources on the web to predict inflation rather than wait every six months for official statistics to be released.
What's next for the project? A senior economist at World Bank has voiced interest in analyzing the data that was scraped and comparing it to WB's data.
Thanks to mobile technology, satellite imagery, and a buffet of online tools, gathering and assessing data has never been easier, and there's never been a more crucial time to do it. The burning question to probe moving forward is, how is big data going to be organized, distributed, and protected -- in business and in activism.