ZIFFPAGE TITLENeed for SpeedBy Tom Steinert-Threlkeld | Posted 2005-07-08 Email Print
WEBINAR: On-demand webcast
Next-Generation Applications Require the Power and Performance of Next-Generation Workstations REGISTER >
Decades before the idea took hold in the dot-com era, Reader's Digest kept a "360-degree view" of each of its customerstracking every contact it ever had with a subscriber to its magazine or a purchaser of any of its condensed books or o
Need for Speed
Even with information so compressed and batches of IBM computers on the job, dealing with tens of millions of subscribers and potential customers for c ondensed books, music and tapes was slow and tedious.
"In the beginning, we blew out the memory'' of the IBM machines, Otten recalls.
Sorting could be difficult. But figuring out how to deal with a cancelled charge could consume lots of code and computational time.
To make efficient use of that time, Reader's Digest came up with a filing system and method of accessing that information that were entirely its ownand still in place until the last pass at customer records stored in the "elegant old lady" took place in May.
Customer records were kept in files of great precision but little flexibility (see chart, p. 46). Each character, each byte, would mean something, in sequence. These long "flat" files might have 25 rows of information, containing everything from a unique identifier for each customer, known as a match code, to the designation of the last mailing of an offer for a cookbook sent to that household. On top of that, each segment, whether it be the master history, the promotion history or the mailing history, could be of vastly different lengths, depending on the activity in each.
Changing the sequence could have disastrous results. Marketing, circulation and payment processing programs all depended on those long files. Each would look for information to appear in a fixed sequence, so that each could draw out just the pieces of information needed, such as the date a payment was received or the mail route that included the household in question.
Even into the 1990s, this was no small matter. The company wanted to begin adding telephone marketing of products, on top of its marketing by mail. But the file had no space allocated to telephone marketing. The sequence of information in the file would have to be changed. All applications would have to be re-trained to look for information in new locations and be able to recognize it when they saw it.
Adapting the system to a new product linePlekoh likens it to "spelunking" in and out of its crevicesmight take three or four months, to change all the records to fit the new data; to make sure all applications that drew on the records knew where to find a new designation and act on it; and to change the hardware.
But in the '60s, the biggest issue was just being able to get at information on customers rapidly, to be able to pull records wanted for a specific marketing campaign and sort accordingly.
An IBM 360 mainframe computer ranged in cost from $831,250 to $34.4 million, in today's dollars. Yet its brain only ticked off 1.33 million tasks a second. Today, a $1,000 IBM ThinkPad laptop ticks off 1.6 billion tasks in the same time.
So, Moore and Otten's programmers made a radical leap to speed up the retrieval of customer information, then stored on magnetic tapes. They would forego the creation or use of IBM 's own application for retrieving data. They would even bypass the operating system. They created a proprietary data retrieval system called the Reader's Digest Access Method that allowed technicians to navigate the customer records. The direct access to the record made it possible to select names for a mailing or do tedious chores such as fixing a problematic mailing label.
Even with a unique and systematic file system and a direct-to-the-data method of grabbing information, managing an electronic library that stored more than 50 million names was not a snap. Updating the Unified File System with new subscribers, changes of address and the like would take four days to complete.
The UFS, though, was cranky, even from the start. Results would be run off five tapes and records would be lost. But the same five tapes could be run again, and different errors would result from the playback. "You know that's kind of scary,'' Otten says. "When you get different results, you know you're in trouble."
A "file maintenance" run, where updates and changes were made to records, would take 125 hours instead of 15 to 20. It took a crackerjack troubleshooter from IBM to find and solve the problem: a logic glitch in the tape drive. "It put 20 years on my life,'' Otten recalls. "It was so frustrating.''
But after the first year, the lady was ready to show off her hex-code-crunching finery. Her real attraction to stat jockeys such as Charlotte Nester would be her ability to determine who was most likely to respond to a given pitch.