Facebook ThreatData
Written by Andrew Johnson   
Friday, 28 March 2014

Facebook has developed a security-focused framework called ThreatData that it says will make it simpler to manage a range of online threats.

The framework attempts to provide a single source of information about threats on the Internet. According to Facebook's Internet Threat Researcher Mark Hammell, the idea behind it is:

“Given the pace of criminals today, one of the hard parts is actually keeping track of all the data related to malware, phishing, and other risks. We wanted an easier way to organize our work and incorporate new threat information we receive so that we can do more to protect people.”

The framework Facebook developers have created lets the company import information about “badness” on the Internet in arbitrary formats, store it efficiently, and making the data accessible for both real-time defensive systems and long-term analysis.

The ThreatData framework has three high-level parts: feeds, data storage, and real-time response.

Feeds collect data from a specific source and are implemented via a light-weight interface. The data can be in imported in most formats, and the feed transforms it into a simple schema that is capable of storing not only the basics of the threat (e.g., evil-malware-domain.biz) but also the context in which it was bad. The added context is used in other parts of the framework to make more informed, automatic decisions.

Once transformed, the data is fed into both Hive and Scuba; Hive is then used to answer questions based on long-term data such as “Have we ever seen this threat before?” and “What type of threat is more prevalent from our perspective: malware or phishing?”, while Scuba is used for more immediate analysis along the lines of “What new malware are we seeing today?” and “Where are most of the new phishing sites?”.

Facebook has also developed a processor to examine the data at the time of logging and act on each of these new threats. Hammell gave examples implemented so far including the fact that all malicious URLs collected from any feed are sent to the same blacklist used to protect people on facebook.com; and that interesting malware file hashes are automatically downloaded from known malware repositories, stored, and sent for automated analysis.

The analysis has highlighted some trends in malware, including a spam campaign aimed at feature phones that was capable of stealing a victim's address book, sending premium SMS spam, and using the phone's camera to take pictures. The framework also lets Facebook view where threats are coming from, arranged by type of attack, time, and frequency. The notes include a worldwide heat map showing malicious and victimized IP addresses, with a pie chart showing similar results for the U.S. by ISP.


In his post Hamell comments:

“Discoveries and detection capabilities like these are just the tip of the iceberg . We’re constantly finding new ways to improve and extend the ThreatData framework to encompass new threats and make smarter decisions with the ones we’ve already identified.”


More Information

Understanding Online Threats with ThreatData

Related Articles

MozDef - Mozilla's Self Defence Kit

Record Payouts At Hacking Contests

Is Exploiting A Bug Hacking?

The Computer Science of Insecurity

Cyber Attacks and Holidays


To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.






or email your comment to: comments@i-programmer.info


CockroachDB 20.2 Adds PostGIS Spatial Data Support

There's a new version of CockroachDB that has updates for developers, better security and new features including support for storing and indexing spatial data using Postgre PostGIS-compatible SQL synt [ ... ]

.NET For Apache Spark Updated

The .NET bindings for Spark have been updated. The new 1.0 version adds support for .NET apps targeting .NET Standard 2.0 or later, as well as support for Apache Spark DataFrame APIs.

More News

Last Updated ( Friday, 28 March 2014 )