Nielsen Builds a Better Platform for Analytics

Today, businesses increasingly live and die by analytics. For the Nielsen Co., which manages massive volumes of data and delivers highly targeted results for clients, the ability to sift through datasets quickly and effectively is critical.

“There are tremendous challenges associated with putting all the data to maximum use,” says Arvind Jade, architecture lead at Nielsen.

The company, which delivers ratings data and other information for businesses in more than 100 countries, faced growing challenges associated with its Global Buy Platform and Answers on Demand (AOD) services. Nielsen provides powerful analytics and reporting capabilities—essentially aggregations on the fly—through an on-demand big data platform. It serves some of world’s largest retailers and others, who track sales of fast-moving consumer goods to drive pricing, product placement, marketing efforts and customer loyalty.

In the past, Nielsen relied on an Oracle database using a custom-built business intelligence layer written in Java J2EE, with a front-end reporting application in Javascript. Metadata was stored in JavaScript Object Notation (JSON) documents.

“We essentially function as a document store,” Jade says. Unfortunately, the system was slow at times, particularly when the metadata underwent changes. “We needed to adopt a more agile approach and achieve gains in efficiency,” he adds.

The company turned to Couchbase, which delivers a NoSQL technology approach that sidesteps many of the limitations of relational databases operating in a multitenant environment. The system delivers about a 50 percent boost in response time by pre-indexing metadata.

“By moving the metadata to Couchbase, we were able to dramatically improve the efficiency of the system and speed data delivery,” Jade says. “We are able to query against the index and target specific documents, something we were not able to do previously.”

In addition, the ratings firm gained more detailed and granular internal reporting tools, including an ability to dive deeper into user patterns and behavior. That has translated into Nielsen designing better products and adapting systems to better fit changing customer buying patterns and behavior.

“We are able to react in a more agile and flexible way,” Jade explains. Moreover, the NoSQL approach has reduced the time required to manage and update systems. “We can use IT staff to add greater value to the business,” he says.

The transition to the system proved relatively seamless. Jade says that the company began using the technology about five years ago, but underwent a major upgrade about a year-and-a-half ago.

The initial transition took about six months, though installing the Couchbase component took less than a month, and the metadata change management process required less than eight hours. Nielsen was able to tap in-house developers and staff with existing Java skill sets to handle the coding and develop the necessary APIs.

Nielsen is now exploring the use of Couchbase Lite, which would introduce a mobile component. Although the current environment is mobile-friendly, it does not yet offer dedicated support for iPads and other tablets and smartphones.

“We are able to put data to work in a more efficient way,” Jade says, “and we have built a strong foundation for the future.”