Dealing With a Data Deluge

With the advent of Web 2.0 and cloud computing, organizations across all industries are drowning in data. While this data explosion presents logistical challenges from a data management perspective, it also offers a wealth of opportunities for organizations that are able to leverage this information effectively.

A number of advanced analytics techniques are emerging in the business intelligence field, but many organizations overlook the most basic?and vital?aspect of successfully leveraging data for a competitive advantage: collecting and retaining all your log data.

At Citizens & Northern Bank, based in Wellsboro, Pa., we use a log management system to ensure the collection and retention of all our syslog data. We purchased our log management suite from LogLogic in 2006 with the intention of using it as a fail-safe for compliance purposes. Since then, we?ve discovered a wealth of other ways to leverage the data stored in our logs and ensure maximum ROI. Here they are:

Streamlining IT operations: One of the greatest benefits of using a log management system is being able to automate the collection of all your syslog data from various systems. We have 24 branches, and all of them have their own routers and switches.

In the past, we had to log on to each system manually and gather all the log information we needed. Now that information is collected in a single, searchable Web-based interface, saving our IT team countless hours in the process. In a typical week, we spend approximately 30 to 60 minutes checking the logs. Without our log management appliance, the same process used to take several hours each week.

Event management: Many executives do not realize that log management systems can also be used for SIEM (security, information and event management). We recently had an event in which our log management system saved our IT team considerable time.

One of our servers started pumping out 14,000 events per minute on the network. Our log management application allowed us to identify the affected server quickly and fix the problem. Otherwise, it would have been a tedious and time-consuming process of searching more than 150 servers. Our log management suite allowed us to restore the network performance to normal levels, and our business operations continued with minimal interruption.

Another challenge for us is low disk space. In the past, we were unable to consistently get to drives running low in disk space in a timely manner, and they would fill up. Now we have a trigger that issues a low-disk-alert e-mail when an event comes in and the disk is near capacity. This allows our IT team to address these issues proactively, rather than after the fact.

Security: Most experts agree that the greatest security threat organizations face today comes from inside their own networks. So it?s necessary to have a security infrastructure that flags any suspicious activity and allows IT teams to quickly scan the entire system and pinpoint the source of any threats.

One of the biggest things Citizens & Northern Bank has done to deal with that is looking at firewall statistics and identifying unusual activity. We monitor traffic going in and out of our firewalls, and our log management appliance gives us custom reports on how many and what types of connections are denied. It gives us a good sense of what is trying to exit our network.

We also monitor VPN connections. A lot of our employees are using broadband thin-client laptops to gain access to our internal network. Now we can see when they connect or disconnect, and even be alerted in real time when they type in the wrong password.

As technology manager, I?m pleased with the ROI Citizens & Northern Bank has seen since deploying the log management suite four years ago. Compliance has never been a headache for us as it has been for many other organizations. We have faith in our logging system and know that whatever data we may need will be there and be readily accessible.

In short, logs allow us to comply with regulations without having to spend enormous amounts of time and money. Additionally, we have been able to leverage our log data in a number of creative ways that are not possible with a traditional SIEM appliance.

Pete Boergermann is assistant vice president and technical support manager at Citizens & Northern Bank, based in Wellsboro, Pa.