The NATO-led Libyan campaign has increased the monitoring of Twitter and other social media in its mission planning, according to the Financial Times. Because there are "too few special forces on the ground", NATO "will take information from every source we can", according to RAF Wing Commander Mike Bracken, the Libyan operation's military spokesman. The article even quotes Twitter user @4libya, who on Tuesday tweeted to @NATO what she claimed were coordinates for Gaddafi forces. Without discussing whether this information was used or not, the article goes on to discuss both the advantages and potential pitfalls of using social media in open-source analysis.
Of the many potential pitfalls of canvassing social media, one barely touched on in the article is the simple fact that there is just too much of it. The firehose of information continues to gush, and any attempt to drink it all will quickly overwhelm a system. As such, an analyst needs a tool that highlights the relevant nuggets, pointing at what needs to be read before anything else.
For our open source analysis, we chose thirty Libya-based Twitter users and collected the past three months of their tweets using a local instance of the Recorded Future platform, which extracts not just entities and events but statements about time. By harvesting this dataset, we were able to see not just who and what the Twitter users were talking about, but when those things were going to happen. This produced some interesting results.
The initial dataset consisted of about 30,000 tweets. Even at only 140 characters each, that's still a lot to read. Of those tweets, 6,840 mentioned a point in time. The overwhelming majority of those time mentions were reporting on things that happened in the past: last night, last week, last Tuesday, etc. However, 144 of those tweets, or about 1.5%, mentioned a time point in the future: tomorrow, next week, next Tuesday, etc. In effect, they were reporting on something that was going to happen. Suddenly, that giant 30,000-tweet dataset has been immediately triaged to a subset that can be read in minutes.
Diving deeper, we were also able to determine not just which users wer tweeting the most often, but which were tweeting most often about events to happen in the future. The following graphic, created in Spotfire, shows tweets per day by user, with future-looking tweets in pink:
Some people tweeted more as time went along, some fell off. Some people started making more predictions over time, some made less, but by visualizing the temporal aspect of discussions, it's possible to focus specifically on one Twitter user's forward looking predictions:
By harvesting a single source and using temporal analytic technology to extract time points, we can easily look at all future-looking statements made by a single user. This can assist in model building, in credibility scoring, and in mission planning.
The analysis of open-source social media can be a great tool to use for defense research, and the NATO mission in Libya shows that forces are attempting to use it. Their first challenge will be determining how to deal with the huge amount of unstructured data in a useful and/or meaningful way.
Chris Holden is a community manager at Recorded Future, an analytics engine that scours the web and analyzes what's been reported about the past, present, and future.