Leveraging Big Data and Cloud for Better Security

As the sophistication of information security breaches continues to grow, organizations realize that they need to do a better job of identifying threats and preventing them from causing data loss and other damage. These companies are exploring a variety of technologies and techniques to bolster their security, including two emerging areas: big data analytics and cloud-based security services.

“Information security needs radical rethinking,” says Steve Wilson, vice president and principal analyst at Constellation Research. “The lessons of data breaches over the past few years are stark Inc. Some of the companies affected by advanced persistent threats and by new hardware attacks were probably doing the best they could.”

Status quo approaches to security are not working anymore, so information security executives need to look for alternative solutions. Many will decide to work with managed security service providers and cloud-based security services, according Jon Oltsik, senior principal analyst at research firm Enterprise Strategy Group (ESG). “Others will bolster their security infrastructure with new controls on end points and networks, as well as better security analytics.”

Clearly, big data—the enormous stores of information that companies are gathering from a variety of structured and unstructured sources—holds much promise as a security tool. And more companies are considering security data as part of their big data efforts.

According to a 2013 report from ESG, 44 percent of organizations surveyed said security data collection and analysis would be considered big data within their organizations, while another 44 percent said they would likely consider security data collection and analysis as part of big data within the next 24 months.

There is a growing volume of security data, ESG says. In the early 2000s, security data collection and analysis focused on network perimeter devices, such as firewalls and intrusion detection and prevention systems. Over time, security analysts expanded data collection to include internal network devices, servers, applications and databases, the firm says.

Newer IT initiatives such as mobile technology, cloud computing and virtualization have added to security data collection. As a result, 86 percent of organizations collect either substantially more or somewhat more security data today than they did two years ago, according to the ESG research.

Using Big Data Analytics for Security

At Automatic Data Processing (ADP), a provider of business processing services for payroll, human resource and other areas, big data analytics for security “represents significant opportunities that we are manically focused on,” says VJay LaRosa, senior director of converged security architecture.

Two key factors have driven the company to use big data for security: the capabilities of adversaries have evolved significantly, and the surge of interconnected devices and the increased reliance on the cloud have created a significant increase in security-related data.

“We are accumulating events at about six billion logs per day in our warehouse, front-ended by a complex event-processing engine,” LaRosa says. “We are actively consuming this data in a relational fashion in a massively parallel data warehouse in order to support the converged nature of our threat management and monitoring programs.”

The data ADP is accumulating comes from security tools, networks and business transactions. As part of its big data security effort, the company is building an 80-node Hadoop cluster for unstructured data storage, with an integrated relational database sitting on top of the Hadoop cluster.

“We are working on developing new real-time streaming analytics with in-memory profiling, coupled with a new Complex Event Processing Engine,” LaRosa says. “We are also working to leverage the built-in, open-source machine learning capabilities that exist in these big data platforms to help advance our capabilities and protect our clients’ funds and data.”

ADP doesn’t disclose the specific products or vendors it’s using, but LaRosa says the company has been using its first-generation platform for about two years. “We are actively building the second-generation infrastructure, which will enable expanded capabilities,” he says. “This new infrastructure will allow us to grow and scale at the size and speeds we need in order to keep pace with this rapidly changing environment.”

The big data analytics efforts have allowed ADP to collect and store raw logs at massive speeds, parse and query the collected logs at speeds supporting critical investigations, and embed analytics into the stored logs to detect malicious patterns or abnormal behaviors.

Neuroscientist reveals a new way to manifest more financial abundance

Breakthrough Columbia study confirms the brain region is 250 million years old, the size of a walnut and accessible inside your brain right now.

Learn More

Picture of Bob Violino

Bob Violino

TRENDING AROUND THE WEB

If you want your 70s to be some of the best years of your life, say goodbye to these 7 behaviors

If you want your 70s to be some of the best years of your life, say goodbye to these 7 behaviors

The Blog Herald

The art of deep focus: 8 habits to get more done in less time

The art of deep focus: 8 habits to get more done in less time

Global English Editing

If you really want to take control of your financial future in 2025, say hello to these 7 new habits

If you really want to take control of your financial future in 2025, say hello to these 7 new habits

Personal Branding Blog

People who express themselves better through text than speaking usually display these 9 behaviors, according to psychology

People who express themselves better through text than speaking usually display these 9 behaviors, according to psychology

Small Business Bonfire

Nature’s prosperity: A symphony of cycles

Nature’s prosperity: A symphony of cycles

The Vessel

If you’re going through challenging times, these 6 habits will make you stronger

If you’re going through challenging times, these 6 habits will make you stronger

Jeanette Brown