Analytics: Reaching Outside Security

Text mining, behavioral analysis may lead to better understanding of attack data, researcher says

AUGUST 2, 2007 | LAS VEGAS -- Black Hat USA -- Can psychological research techniques help you understand what's happening in your network? How about customer data mining techniques? One researcher suggests that these disciplines -- and many others -- can help with security analytics. In a presentation at the Black Hat conference here today, Mark Ryan del Morai Talabis, an analyst for Secure DNA, said researchers and other security pros should be looking beyond their off-the-shelf tools as they scrutinize attack data and look for trends and patterns. "One of the big problems in security right now is that there are so many data collectors," says Talabis, who also works with the Honeynet Project. "It's like having a diamond in the rough. The data could be very valuable, but we need better ways to analyze and polish it." The Honeynet Project recently has completed the Global Distributed Honeynet, which links a number of honeypots across the globe, Talabis notes. "It's bringing us tons of data," he says, "and that's one of the reasons I've been looking at some more creative methods to study it." The Honeynet Project has developed Honeysnap, a tool that helps structure security and attack data to make it easier to manage, he notes. But there are a wide variety of other analytics tools that may also yield results for security research, and many of them are open source applications that can be found for free on the Web, Talabis observes. One such product is YALE, also known as RapidMiner, a tool originally designed to help with data mining for marketing professionals. "You can use YALE to do text mining of hacker chatter, to help identify patterns and trends in the conversation," Talabis says. "You can also use it for clustering attack data, to show how the traffic is grouped." Security pros and researchers can also use statistics tools that were originally designed for economic analysis, Talabis suggests. "These types of tools have been used to study game theory, where you see how one side moves and the other responds, and how over many games, there's an equilibrium," he says. "That same idea applies in security, where the attacker moves, and the defender responds, and so on." Even the discipline of psychology may have some tools that are useful to security analysts, Talabis says. "You can use tools designed for behavioral analysis to study the patterns of behavior among hackers in the chat rooms." The key is not to be constrained by the tools that are available in the security industry, Talabis says. "If you look into other fields, you may find your options are a lot broader than you think."