Big Data by the Numbers

As organizations collect more data, figuring out how to store, structure, and use it is a growing challenge.

Big data is rapidly moving into the mainstream of business. The ability to sift through mountains of structured and unstructured data and put it to maximum use increasingly separates industry leaders from laggards.

“Companies not only have to capture and store data they must be able to render it and make sense out of it in the context of business issues,” says Gary Curtis, chief technology strategist at Accenture. “Because of widely proliferating media types and numerous ways of using data, the challenge is significant.”

A recent survey conducted by LogLogic, an enterprise-class log management infrastructure and analysis provider, found the amount of IT data organizations are producing is growing exponentially—with 62 percent producing more than 1 terabyte and up to 1 petabyte of data.

Unfortunately, a serious lack of understanding exists about big data. A hefty 38 percent of respondents—comprised of more than 200 security and IT operations professionals—said they lack a clear understanding of what big data is. In addition, 59 percent do not have the tools to manage data from their IT systems and 72 percent are unable to manage data in the cloud.

“These survey results show that organizations collect many terabytes of IT data. However, they are not taking advantage of the intelligence this data contains,” says Mandeep Khera, CMO for LogLogic. He says that it’s akin to “holding the key to a safe but never opening it…companies have access to valuable insights but have not taken the steps to unlock these insights.”

In fact, many organizations rely on antiquated and disparate methods, processes and tools to try to manage all the data. The widespread use of spreadsheets contributes to the problem. A central viewpoint as well as sophisticated analytics capabilities are critical to making sense out of big data.

“Without a central viewpoint with intelligent analysis, orgs are unable to derive actionable insights,” Khera says. “While a lot of people are talking about big data and cloud log data, very few of them understand what it takes to manage it. Companies who do so will gain a competitive advantage. ”