WestLB Tackles Electronic Discovery

By Kim S. Nash  |  Posted 2006-08-02 Print this article Print

When encountering legal or regulatory action, technology managers who fail to get corporate data fast or vouch for its completeness can cost their companies millions of dollars. Learn what happened to WestLB, an investment bank, when it had to exhume 650,

Just Like You

Like any investment bank, WestLB advises clients on which international stocks to buy and sell. The 37-year-old company is based in Düsseldorf, Germany, but runs large offices in London and New York. Privately held, WestLB hit hard times recently as the German economy sagged. After reporting $5.1 billion in losses from 2002 through 2004, the bank wanted to cut costs by, among other things, outsourcing global technology operations to Hewlett-Packard.

The five-year, $500 million deal in 2004 immediately saved WestLB overhead, in part because the bank was able to shift 450 technology employees—almost all of its systems staff—to Hewlett-Packard. HP took over desktop computers, e-mail, telecommunications and networking, customer support and Web activities for the bank in January 2005, 3 1/2 months after Quinby filed her lawsuit, according to court filings. The vendor is not a defendant in the suit.

To help manage the outsourcing transition in the U.S., Ken Bigelow, a six-year veteran of WestLB in New York and head of trading technology in that office, was promoted to CIO of the New York office in November 2004, he said in his deposition. He was the third CIO there in three years.

Bigelow described WestLB's e-mail systems in his deposition last summer.

At the center is Lotus Notes, the second most popular mail system in the world, behind Microsoft Outlook, says messaging market researcher The Radicati Group. All WestLB employees use Notes for everyday e-mail, and the Notes servers are copied every night to magnetic tapes.

Such tapes are the standard for backing up data at large companies, according to Todd Stefan, principal at Setec Investigations, a computer forensics firm in Los Angeles. "Tape is used everywhere," he says.

Quinby and the rest of WestLB's sales team used Notes, but their messages were archived differently, according to Bigelow's deposition and affidavit, because of the SEC's requirement to save their e-mail for three years. For the first two years, it must be "readily accessible," meaning the company or outside auditors must be able to retrieve it from a live server. Archival backup tapes warehoused offsite at a disaster recovery facility don't qualify. At WestLB, in addition to nightly tape backups of the Notes servers, broker e-mail was also saved to optical disks via AXS-One's namesake archiving package. Countrywide Financial, Fannie Mae, and MONY Group are also AXS-One customers, according to the vendor's Web site.

In a setup like WestLB's, AXS-One would run on the same Sun Solaris server as Lotus Notes, with a C++ interface to the e-mail program, says Marie Patterson, vice president of marketing at the Rutherford, N.J., vendor. She declined to talk about WestLB specifically, but explained that after someone hits "send," AXS-One holds the message at the e-mail gateway so it can save a copy of it, plus any attachments, before allowing it to pass to the recipient.

While tape backups snare whatever happens to exist on the Notes server at the moment the backup routine launches, AXS-One keeps in a storage system a single copy of every message created by employees whose mailboxes are connected to it.

WestLB also uses the instant messaging function built into the Bloomberg terminals that bring stock market and other financial data to subscribers. Bloomberg often archives instant messages for customers.

Bigelow also recounted WestLB's other archiving systems and procedures.

Every night, backup utility software automatically copies the company's entire Notes server onto 20 to 40 magnetic tapes. At the end of a month, one of these nightly snapshots—usually the one from the last day of the month—is designated WestLB's monthly backup. Daily archives are kept at HP or the bank's disaster recovery vendor, Iron Mountain, for 15 weeks, monthlies for 13 months. Then, they are overwritten as the tapes get reused after 15 weeks or 13 months.

At the end of a year, a backup from late December is designated the backup for that year. WestLB keeps annual snapshots indefinitely.

WestLB's backup tapes aren't meant to be a reference library of 365 days of electronic activity, Bigelow said. They're intended to capture a reasonably complete set of data at a moment in time, to be used to rebuild servers after a disaster. They wouldn't, for example, catch an e-mail exchange between two people at 10:30 p.m. if both of them immediately deleted the messages before an 11 o'clock backup.

From the tapes and disks it had, WestLB technology managers scrambled to satisfy Quinby's data demands. But the legal review of the resulting torrent of documents "paralyzed us for a significant period of time," said WestLB lawyer Groman Darringer in court papers. The bank's lawyers and technology people worked full-time on the case for weeks.

Next Page: A 'Boys' Club'?

Senior Writer
Kim has covered the business of technology for 14 years, doing investigative work and writing about legal issues in the industry, including Microsoft Corp.'s antitrust trial. She has won numerous awards and has a B.S. degree in journalism from Boston University.

Submit a Comment

Loading Comments...
eWeek eWeek

Have the latest technology news and resources emailed to you everyday.