Wiland Boosts Performance With Log ManagementBy Samuel Greengard | Posted 2015-10-16 Email Print
A marketing analytics firm adopts a sophisticated open-source log management solution to boost IT performance and identify problems significantly more quickly.
An emerging truth in the digital age is that just about every business process or practice intersects with analytics. As organizations look to boost market intelligence and overall performance, they're turning to internal capabilities and outside services to take performance to a new level.
One company that's in the middle of this revolution is Wiland, a Niwot, Colo., a marketing analytics and intelligence firm that specializes in extracting data for retailers, publishing companies, consumer services firms, nonprofits and the travel land hospitality industry. To achieve those objectives, the firm must transform data into information and business intelligence, and that requires a robust IT infrastructure, as well as the right combination of analytics tools and other software.
Wiland, which relies on a Hadoop cluster and 15 database servers in its production environment, was facing formidable challenges, particularly in the critical area of log management. A previous free version of a commercial product couldn't meet the company's needs, but the full version of the software was too expensive. Nevertheless, "The logging data is critical for maintaining security and overall IT performance," says Eric Puening, Wiland's director of IT infrastructure.
The company has strict requirements about how it handles, stores and parses data. "We have a lot of sensitive data for companies, so we have to make sure we are adhering to guidelines and standards," Puening explains.
Powerful, Affordable Log Management
As a result, Wiland turned to open-source log management vendor Graylog for a solution that was both powerful and affordable. The software offers a unified platform for centralized log management; a multi-tier architecture that separates ingestion, processing and storage; and real-time processing of log data prior to storage.
"Because we run our production stacks on open-source technology, it was a good fit," Puening says.
Wiland began installing the software in late 2014 and went live with the implementation in the spring of 2015. It is continuing to deploy the system across the company, and the firm is already seeing solid results.
For example, when Wiland's IT team couldn't figure out why network shares or backup systems would not connect to the Hadoop cluster, they quickly identified a global access control list (ACL) rule that was denying the connection.
In another case, after Wiland moved some of its primary file systems and split the files across two network file system (NFS) servers to better handle the data load, performance lagged. Checking the logs, the IT team was able to detect that one of the servers was still making requests at hundreds of messages per minute to a mount that didn't exist.
"We are able to find the proverbial needle in the haystack when it comes to identifying problems in the vast noise of logs," explains systems administrator David Peterson. In some cases, the time savings amounts to hours. He estimates that in a six-month span, the solution helped the company recoup about a full week of staff time.
Although setting up the system and rightsizing it presented some challenges, the organization is now on solid ground—and frequent releases have been beneficial. "The updates are frequent and robust," reports systems administrator Saulius Zilis.
The next step, he says, is to add logging for Windows and firewalls. "Our log management has taken a giant step forward, and that benefits the business," Zilis adds.