By Tom Steinert-Threlkeld  |  Posted 2005-07-08 Print this article Print

Decades before the idea took hold in the dot-com era, Reader's Digest kept a "360-degree view" of each of its customers—tracking every contact it ever had with a subscriber to its magazine or a purchaser of any of its condensed books or o

Confusion Matrices'">

'Confusion Matrices'

The new clearinghouse of information held on Reader's Digest's computers—or those of its outsourcer—would have an equally straightforward and forgettable name: the Marketing Systems Replacement, or MSR.

This attempt to move off the Unified File System was pitched in late 2002 by Regan, now vice president of database marketing services, and Hilliard, who came from Andersen Consulting during the push to introduce pieces of the previous big effort, CIMS.

In that campaign, the first idea was to convert to a wholly new system in a single "big bang." Of course, any conversion, then or now, would be like "flying a 747 across the ocean and changing the engines in mid-air,'' says the company's chief information officer, Jeffrey S. Spar.

Most critically, in the '90s, the UFS handled both marketing, to generate new orders; and fulfillment, where books, magazines, tapes and the like were sent out to meet those orders.

But after Reader's Digest went public in 1990, its conservative readership began to literally die off. The company compounded the problem by flooding its market with direct mail. A new CEO lasted only three years. Thousands of workers were cut loose, including hundreds in Pleasantville. Net income of $264 million in 1995 dropped to $49.5 million last year. Revenue, topping $3 billion in 1996, dropped below $2.4 billion last year.

Grappling with the slide, the company decided its core competitive advantage was how it used information to market to customers. The fulfillment of orders would be contracted out, to Communications Data Services in Des Moines, Iowa, starting in May 2000.

Information on those orders would come back from the fulfillment house and be kept in the Unified File System, ensuring that the view of customer activity remained complete. The ability to send out magazines, books and other products would not be affected by any system downtime—or changeover.

If, during the changeover, selections of new marketing targets didn't work properly, that was not a critical problem, as long as the disruptions didn't get out of hand.

Through 2004, Hilliard—who had become the leader of the replacement project—and Regan were trying to transfer UFS files into tables in the new system. They figured they could transfer data a bit at a time, then update those records directly in MSR.

But the systems would not sync easily. The old system was accurate only to the month; the new one, to the day, Regan says. If a conversion was off by only one-thirtieth of an answer, it was the wrong answer. There were other miscalculations, such as double-counting the write-offs attributed to some customers who never paid their bills.

The biggest bugaboo? That most reliable predictor of customer readiness to respond, Promotions Since Last Order, was not working.

For inexplicable reasons, the PSLO, which should have a value between 2 and 4, kept "zeroing out,'' according to Regan.

"Every day, it was, 'Can't we get a break?'" she says.

Hilliard would stay on e-mail and instant messaging until 2 a.m. on Saturday nights, trying to fix the problems.

They had to redo the PSLO calculation several times. But that was just one term that had to be right. There were 1,400 terms, plus 300 equations that used the terms. Each had to be translated to run properly in the MSR.

Regan and Hilliard started plotting results of the new system against those of the old system. Results of the old system would be depicted on a straight line from the bottom right to the upper left of a graph. Results of the new system would be plotted wherever they fell. If they all fell on the line, fine. If not, check it out.

Take the Individual Age matrix. Reader's Digest reset any birth that was before 1900—at 1900. But the logic of the replacement system did not carry out the 1900 rule accurately. Too many subscribers and customers appeared to be more than a century old. Only by repeatedly reviewing the raw data in both systems could the confusion be eliminated.

Regan and Hilliard would run these "confusion matrices" thousands of times, before being satisfied the new system would work as well as the old lady.

Her time was almost up. The cost: By one estimate, nearly $1 billion had been spent on hardware, software, consulting services and staff time since the advent of CIMS in 1988. The reality: No one knows. "Those unkind have said 'Countless Innumerable Millions Spent,'" Burns notes.

But the decision on whether the new system was ready to support a business did not fall to Regan or Hilliard. That was the statisticians' call: Nester; Al Virgo, the database marketing director for books and music; and Aiyi Wang, associate director of database marketing. After all, finding the right names to send marketing appeals that produce orders is what matters.

Ultimately, it was Nester who made the call. On April 27 she gave the high sign that the new terms were working and the replacement system ready to perform as the database of record.

The first bulk mailing, a music campaign, was hard. "Really hard," Regan says. There was no template to follow or copy over from a prior campaign. Every step had to be checked thoroughly, from the elimination of names to the equations used. The elegant old lady had made sure that every name ended up in only one box, the target of only one mailing in a campaign. In the new system, a name could end up in more than one box. Selections from box A had to be run against box B to make sure there were no duplications—a time-consuming task. Where database marketers like Virgo had gotten used to having selections made two weeks in advance of when they had to be delivered to a letter shop, this mailing went down to the wire.

In fact, the first bulk mailing in MSR was completed in the second week of May—just as Devanny was enunciating his eulogy for the original one, the Unified File System.

"The fact that a 35-year-old system could support a business that well ... I'm amazed at the architecture. That's kind of magical,'' says Regan, who retired on May 20. "Hopefully, [the new system] will be as robust and successful.''

Or, at least, live as long. Whether MSR will be as effective remains to be seen. Also in doubt is how much it will change. No system runs forever—or untouched.

"It's a constant challenge to keep them up-to-date; it's a challenge to make sure that they're accurate, make sure that they're doing what you think they're supposed to be doing,'' Regan points out. "These systems are never done," she says.

Except for the elegant lady. Now retired, too, having taken no time off since Nov. 16, 1969.

Tom was editor-in-chief of Interactive Week, from 1995 to 2000, leading a team that created the Internet industry's first newspaper and won numerous awards for the publication. He also has been an award-winning technology journalist for the Dallas Morning News and Fort Worth Star-Telegram. He is a graduate of the Harvard Business School and the University of Missouri School of Journalism.

Submit a Comment

Loading Comments...
eWeek eWeek

Have the latest technology news and resources emailed to you everyday.