How to Get Real-Time Analytics from a Data Warehouse

It’s a cliché in business to say that time is money, but the people who repeat it usually don’t quantify how much of one it costs to buy the other.

That’s probably a good thing, too, because the equation is changing. Time is getting cheaper.

Not radically cheaper: Saving time for other parts of the business still costs IT hard dollars for compute power, lines of code and bandwidth.

But the kind of technology that can crunch and deliver data quickly enough that its results are considered “real-time” has become so much less esoteric that it is now available to more than just brokerages, air-traffic controllers and emergency response agencies—organizations that would lose either their clients or their shirts if their information were more than a few seconds old.

Many of the companies moving into real-time systems are those for whom it’s a priority to measure time in seconds and to preserve as many as possible. Others are more surprising. Grocery products, for example, come with notoriously narrow profit margins, short shelf lives and complex buying patterns that have made after-the-fact data mining more successful than real-time sales tracking.

But near-real-time data allows grocery chain Haggen, Inc. to respond to low-inventory warnings as the inventory dwindles, rather than long after both inventory and prospective customers are gone.

“The software in our stores used to serve up summary files and log data at 3 a.m. We’d pull it across from all our stores by about 6 a.m. and begin the process of loading it into our data warehouse. By about 9 a.m., the business would begin to get some idea of what had gone on the previous day,” according to Harrison Lewis, CIO of the 33-store chain, which is based in Bellingham, Wash.

“Now we have a trickle-feed of data coming in all the time from the stores. Within fractions of a second after the transaction we can pull the data out, convert it to XML, send it to the controller and on to the server,” Lewis says. “We get visibility throughout the day; at 9 a.m. I can say how I was doing 15 minutes ago.”

That’s important for a store that focuses on high-quality, local products and—more importantly—products being mixed or baked or cooked in the store and sold for high margins in the company’s specialty departments.

Since we’re “dealing with products we’re manufacturing in the store, or with items that are doing incredibly well, having the ability to respond faster gives me the ability to take advantage of a good situation or minimize the impact of a bad one,” Lewis says.