Leveraging Digital Data Streams and Analytics

By Madeline Weiss  |  Posted 2012-12-18

By Madeline Weiss

Digital data are being created at the rate of 2.5 to 3 quintillion bytes daily, according to such disparate sources as IBM and Dilbert. The data are created from customer transactions; company operations; vehicles; consumers using the Internet; digital cameras; sensors on machines, inanimate objects; and animals—the list goes on. Enterprises need to pay attention to this deluge of data because it creates storage challenges that must be deal with, and this data can be a source of company value.

The CIO members of the Society for Information Management (SIM) Advanced Practices Council (APC), a forum for senior IT executives, have been learning how to harness real-time digital data streams (DDS) to increase revenues and profits, gain operational efficiency and achieve competitive advantage.  

At the May 2012 APC meeting, they met with Bryan Mistele, founder of INRIX, a company that forecasts traffic conditions by combining real-time data from more than 600,000 GPS-enabled vehicles with data from multiple private and public sources (e.g., road work, accidents, events, policy activity, historical traffic patterns).

INRIX operates in more than 30 countries and powers navigation devices for major automobile manufacturers worldwide. It delivers traffic broadcasts globally and provides traffic advice through Android and iPhone apps. The forecasting engine that analyzes the data in real time is a sophisticated set of algorithms that's capable of accounting for dozens of variables (including weather conditions, construction schedules, holidays, sporting events and historical traffic patterns) to provide more than 100 million people with real-time predictive traffic information.

INRIX's story is not only about creating a successful new company by harnessing real-time digital data streams along with predictive analytics. It's also about creating new value for existing companies.

At this same meeting, APC members met with researchers Gabriel Piccoli and Federico Pigni, who presented these five value-creation archetypes for extracting value from DDS (Harvesting External Data: The Potential of Digital Data Streams, Gabriel Piccoli and Federico Pigni, Advanced Practices Council publication, 2012).

·        DDS generation

·        DDS aggregation

·        Service

·        Efficiency

·        Analytics

DDS Generation

The firm originates the stream knowingly or unknowingly. TripIt generates a simple, mobile travel itinerary with travel details from an individual's flight, hotel and rental car confirmation emails that can be accessed either online or from a mobile device. It streams the itinerary to partners that can harvest it and create value-added services based on the TripIt platform. For example, Expensewatch.com automatically computes expense reports.

DDS Aggregation

The firm collects, aggregates and repurposes DDS. INRIX aggregates data from multiple sources to create its predictions. Socrata.com aggregates federal, state and local data, making it available in one-stop fashion to the public.


The firm uses one or more digital data streams to provide additional or improved services to consumers. INRIX provides traffic predictions, and MyCityWay.com aggregates hundreds of datasets and DDS (originating from other entitles, such as government sources and the restaurant intermediary OpenTable) to provide consumers with a convenient menu of real-time services available nearby. Similarly, mytaxi.net provides its reservation and taxi services mainly by collecting the GPS coordinates of both taxis and customers through a mobile app.

JetBlue monitored Twitter streams that mentioned JetBlue during the 2010 holiday season when "Snowpocalypse" hit New York. Axel Murillo was trying to get to Austin and couldn't get through to JetBlue phone lines due to the heavy phone traffic. He tweeted about it, mostly to complain about his bad luck. On Twitter, a Jet Blue representative asked him to direct message his confirmation number. Eighteen minutes later, the agent replied via direct mail that he had been rebooked.

When that flight was cancelled the next day, Murillo went straight to Twitter and received a similar response from another representative. It happened a third time, with the same result. A few days after Murillo's return to New York, he received the following email from a Jet Blue officer: "As a token of our appreciation for your patience during last week's snowstorm when we cancelled your flight, please accept 10,000 TrueBlue points, which you can apply toward future travel to any JetBlue destination."


Digital data streams are used to optimize internal operations or track business performance. One example is ruter.no, which was developed for Oslo, Norway, to gather real-time information on the overall state of the city's public transportation system, while also providing a convenient source of public transport information.

The transit authority also uses this traffic information to optimize the traffic signal priority system, thereby improving travel times, reducing the number of vehicles and lowering the costs of transportation. The company estimates that the average driving time of buses on some of the heaviest bus lines was reduced by up to 20 percent.


The firm uses DDS to develop superior insight or knowledge in order to enable better decision making. Predictive analytics is at the heart of INRIX's offering.  

Mint.com, now owned by Intuit, provides its seven million customers with convenient visual tools to examine the status of their expenses and investments. The data comes from customers' bank, credit and investment accounts. Mint.com also offers recommendations for meeting financial goals, such as cutting down on rent, gas, clothes and latte expenses.

Based on Piccoli's and Pigni's broad survey of CIOs, they concluded that opportunities to exploit DDS are large and mostly untapped. Currently used for monitoring the business, DDS and predictive analytics also have the potential to transform customer service.

The greatest challenges in reaping such benefits are integrating DDS, recruiting staff with skills in data science and analytics, and working across organizational silos to change the way people think and behave.

Madeline Weiss is the director of the Advanced Practices Council at the Society for Information Management (www.simnet.org/).