Leveraging Big Data and Cloud for Better Security

By Bob Violino  |  Posted 2014-01-30
big data and cloud security

As the sophistication of information security breaches continues to grow, organizations realize that they need to do a better job of identifying threats and preventing them from causing data loss and other damage. These companies are exploring a variety of technologies and techniques to bolster their security, including two emerging areas: big data analytics and cloud-based security services.

"Information security needs radical rethinking," says Steve Wilson, vice president and principal analyst at Constellation Research. "The lessons of data breaches over the past few years are stark Inc. Some of the companies affected by advanced persistent threats and by new hardware attacks were probably doing the best they could."

Status quo approaches to security are not working anymore, so information security executives need to look for alternative solutions. Many will decide to work with managed security service providers and cloud-based security services, according Jon Oltsik, senior principal analyst at research firm Enterprise Strategy Group (ESG). "Others will bolster their security infrastructure with new controls on end points and networks, as well as better security analytics."

Clearly, big data—the enormous stores of information that companies are gathering from a variety of structured and unstructured sources—holds much promise as a security tool. And more companies are considering security data as part of their big data efforts.

According to a 2013 report from ESG, 44 percent of organizations surveyed said security data collection and analysis would be considered big data within their organizations, while another 44 percent said they would likely consider security data collection and analysis as part of big data within the next 24 months.

There is a growing volume of security data, ESG says. In the early 2000s, security data collection and analysis focused on network perimeter devices, such as firewalls and intrusion detection and prevention systems. Over time, security analysts expanded data collection to include internal network devices, servers, applications and databases, the firm says.

Newer IT initiatives such as mobile technology, cloud computing and virtualization have added to security data collection. As a result, 86 percent of organizations collect either substantially more or somewhat more security data today than they did two years ago, according to the ESG research.

Using Big Data Analytics for Security

At Automatic Data Processing (ADP), a provider of business processing services for payroll, human resource and other areas, big data analytics for security "represents significant opportunities that we are manically focused on," says VJay LaRosa, senior director of converged security architecture.

Two key factors have driven the company to use big data for security: the capabilities of adversaries have evolved significantly, and the surge of interconnected devices and the increased reliance on the cloud have created a significant increase in security-related data.

"We are accumulating events at about six billion logs per day in our warehouse, front-ended by a complex event-processing engine," LaRosa says. "We are actively consuming this data in a relational fashion in a massively parallel data warehouse in order to support the converged nature of our threat management and monitoring programs."

The data ADP is accumulating comes from security tools, networks and business transactions. As part of its big data security effort, the company is building an 80-node Hadoop cluster for unstructured data storage, with an integrated relational database sitting on top of the Hadoop cluster.

"We are working on developing new real-time streaming analytics with in-memory profiling, coupled with a new Complex Event Processing Engine," LaRosa says. "We are also working to leverage the built-in, open-source machine learning capabilities that exist in these big data platforms to help advance our capabilities and protect our clients' funds and data."

ADP doesn't disclose the specific products or vendors it's using, but LaRosa says the company has been using its first-generation platform for about two years. "We are actively building the second-generation infrastructure, which will enable expanded capabilities," he says. "This new infrastructure will allow us to grow and scale at the size and speeds we need in order to keep pace with this rapidly changing environment."

The big data analytics efforts have allowed ADP to collect and store raw logs at massive speeds, parse and query the collected logs at speeds supporting critical investigations, and embed analytics into the stored logs to detect malicious patterns or abnormal behaviors.

Evaluating Cyber-Threat Detection

Also exploring big data for security is Health Care Service Corp. (HCSC), which operates Blue Cross and Blue Shield of Illinois, Montana, New Mexico, Oklahoma and Texas. The company is in the midst of a research-and-development project focused on evaluating big data and advanced data visualization technologies for potential applications to cyber-threat detection.

"Our team has done an extensive amount of research in this area over the last four years," says Tom Baltis, executive director, IT Governance, Security and Risk Management at HCSC. The company expects to launch a big data/visualization security initiative within two months, and, by the end of this year, it hopes to have a fully functional solution in place throughout the enterprise.

"We already have a very sophisticated set of capabilities for analyzing security events and detecting cyber-security incidents, but we're looking to innovate in this space by applying big data technologies," Baltis reports.

By using business intelligence (BI) and data visualization tools applied to various types of data the company gathers, HCSC hopes to be able to detect types of incidents that traditional security tools are not able to find—or are even aware of. Its priority is to enhance real-time or near-real-time incident detection.

HCSC also aims to detect security incidents based on data that it has accumulated over the past weeks and months, by applying data analysis tools. By analyzing this data and translating that analysis into automated rules, the company hopes to be able to more quickly detect similar incidents when they occur, and even to predict future security threats and prevent them from causing damage.

"We're always looking for more effective ways of identifying security incidents—trying to become better at what we do and discover things we don't see with traditional tools," Baltis explains.

As far as which technology products HCSC will use for its big data/visualization effort, the company is "casting a broad net," Baltis says, exploring a variety of commercial and open-source BI reporting and processing products and services.

"In the end, our solution will most likely comprise a number of tools, and it might involve some custom-built tools," Baltis says. He says the company aggregates extensive amounts of data, and that data will be continually reviewed and analyzed to detect patterns that may indicate a cyber-security incident.

Turning to Cloud-Based Security

Companies are also investing in cloud-based security services to thwart attacks.

In January 2014, Northrop Grumman, a global aerospace and defense technology company, began using CA CloudMinder, a software-as-a-service (SaaS) offering from CA Technologies that provides user provisioning, self-service user management and an access request system; risk-based authentication; and federated single sign-on for both cloud-based and on-premise applications.

"With identity federation and single sign-on, users can enter a single password to access any application—whether on the customer's government or company-owned cloud or on Northrop Grumman's cloud infrastructure," says Zaki Saleh, business development director, Health IT, at Northrop Grumman.

The company's federal and state health and human services customers "are faced with increased demand from their constituents to access government services online," Saleh explains. "As demands increase, our customers are looking for identity and access management [IAM] solutions to allow constituents to access services through multifactor authentication."

Once a constituent is authenticated, there are subsequent needs for managing access and authorization for specific services or applications, Saleh adds. CA CloudMinder, which can be deployed either on-premise or on a private or public cloud, will help address those needs, he says.

These and other emerging technologies are helping organizations around the world deal with growing and ever-more-virulent security risks.