Security Alert: When Bots Attack

 
 
By Kim S. Nash  |  Posted 2006-04-06
 
 
 

The malicious code snuck through Auburn University's firewall and onto one of the school's lab PCs in an electronic message. On that September day in 2004, Auburn's network security specialist, Mark Wilson, watched from his computer what happened next.

The message contained a link, an invitation to visit a Web site that the PC's owner, possibly a student, found too enticing to resist. He or she couldn't know that clicking this link would download dirty code, letting it burrow into the PC through an unpatched bug in the Microsoft Windows operating system. It wasn't a straight Trojan or a worm, but a combination of programming malice with far greater potential for harm. It would allow hackers to seize control of the machine and turn it into a "bot," a remote-controlled robot that they could order to send spam or steal data and, most important, turn other vulnerable computers on the university's network into bots just like it.

But the trick worked. Click. Immediately, the Alabama university, like so many other colleges, companies and government agencies, fell victim to what security experts call one of the biggest cybersecurity threats out there: bot attacks. Auburn's network was thrown open to hackers all over the world.

With the back-to-school assault on Auburn, whoever launched the attack was probably after more computers to enlarge his or her botnet, Wilson says. The code exploited a bug in Windows' LSASS, or Local Security Authority Subsystem Service, which is how Microsoft verifies users who log on to the Windows 2000 or Windows XP operating systems. Though Microsoft had released a patch five months earlier, not all of the computers at the

23,000-student school were updated, Wilson says.

Auburn, of course, isn't the only organization to be hit by bots.

On any given day, 3 million to 3.5 million bots are active around the world, says Alan Paller, director of The SANS Institute, a security researcher in Bethesda, Md.—enough to disable all U.S. online retailers three times over. And each day those bots infect 250,000 Internet Protocol addresses, representing hundreds of thousands of Internet-connected devices, according to CipherTrust, a security consultant.

While more than 50% of bot attacks go after home PCs, CipherTrust has also found bots in 40% of large and midsize companies. Caterpillar, CNN, eBay and Microsoft are among the companies that have suffered bot attacks.

And the threat is growing. The number of new successful bot strains—variants in bot code—was up 538% last year alone, says another computer security company, Cybertrust, which tracks the activities of 11,000 hackers and works with the Federal Bureau of Investigation on cybercrime cases.

Consider the government's recent, high-profile case against Jeanson James Ancheta, a 21-year-old hacker from Downey, Calif. Ancheta, who holds a high school equivalency diploma, pleaded guilty in January in U.S. District Court in Los Angeles to building and selling bots, using the profits to build his business, and using his network of thousands of bots to commit crimes.

According to the plea agreement, he had made more than $60,000 and infected at least 400,000 computers, including machines at two U.S. Department of Defense facilities. He also provided bots for intrusion by others. One potential target was electronics giant Sanyo, which declined to comment. So did a spokesman for the DOD's Joint Task Force-Global Network Operations, who refused to discuss details of Ancheta's attack "for security reasons."

"It's a constant battle," says Michael Lines, chief security officer at credit reporting firm TransUnion, the consumer credit report company in Chicago. Lines says TransUnion, with its one terabyte of sensitive financial data, is a frequent bot target, though he will not provide details. "There is no single technology or strategy to [solve] the problem," he says.

Bots may disappear as people clean up their PCs and patch their software so malicious code can't get in, but they are quickly replaced by new bots adapted to exploit different problems, including "zero-day exploits," software bugs for which patches don't yet exist.

One reason bots are such a troubling security concern is that hackers don't have to build their own code to create the intruders—they can download bot toolkits for free on the Internet.

They can even buy access to bots. Ancheta linked a price list to his "botz4sale" online channel, according to the plea. He offered up to 10,000 compromised PCs at a time on the underground hack market for as little as 4 cents each.

Some bots cost more. A PC on a government network, for example, may sell for as much as $40, according to CipherTrust, because it offers access to loads of potentially interesting information. Bots that attack brand-new exploits are also considered more valuable.

Once a bot is created behind a corporate firewall, the person who controls it can mess with company applications by, for example, installing a keystroke logger on the PC to capture passwords as they are typed.

Or by exploiting the right application or operating-system bug, a botmaster can copy, manipulate or delete customer information, personnel records or almost any data on the infected machine.

In Israel, Ruth and Michael Haephrati, age 28 and 44, pleaded guilty in March to several conspiracy and computer crimes involving bots, according to published reports in ComputerWeekly and in Globes, an Israeli news service. They built spying software that they sold to Israeli competitive intelligence companies, which snuck it onto vulnerable computers at their clients' competitors, illegally gathering corporate information, according to the reports.

Ancheta, on the other hand, used his botnet for other moneymaking ventures. He sold or rented bots to people looking for computer power to send spam, or launch denial-of-service attacks to disable specific Web sites, according to his plea. He made a few hundred dollars from each deal.

Like a legitimate technology vendor, Ancheta provided consulting help with his product. Tips included how to perform bits of mischief such as a "synflood," to take out a Web server by flooding it with bogus requests to connect, according to his plea.

More lucrative for Ancheta was defrauding online advertising companies. Adware companies will pay "partners" for each digital advertisement they install on a PC. The adware monitors the user's activity, such as what terms he searches for at Google or Yahoo, and then displays related pop-up ads. Sometimes ads will just play across the screen, unrelated to anything the user is doing.

Above-board adware partners can, for instance, bundle adware with other software they sell, such as games or screen savers. But when botmasters play this game, they instruct their bots to install ads on machines they've taken over, collecting as much as 40 cents for each successful placement. They sometimes clog a PC so much it can't function.

Check stubs, bank records and files from online payment service PayPal seized by prosecutors show that Ancheta and an unindicted co-conspirator, someone indentified in court papers as a juvenile nicknamed "SoBe," took in $58,357.86 this way in less than 12 months.

In an AOL Instant Messenger conversation between Ancheta and SoBe that was archived in files seized in the case, Ancheta said of the money he made from adware, "It's easy, like slicing cheese." But the cash flow depended on keeping his botnet strong and growing.

Exactly how Ancheta got his bots into computer systems is not known. Some court records so far are sealed, the companies and government agencies named in the case won't talk, and neither will Ancheta's lawyer, who did not respond to Baseline's request to talk with Ancheta.

Attack In Progress

To gain an understanding of bots and botnets, what happened at Auburn University serves as a good example of how these attacks occur. In fact, according to FBI Supervisory Special Agent Kenneth McGuire, the Auburn incident had "all the earmarks of [Ancheta's] type of activity." The Auburn bots, which were based on code called Rbot/Rxbot, sought out the specific LSASS weakness in the Windows operating system. In addition, Auburn's records of the attack show malware coming from a Web site with the address resili3nt.superihost.com. Ancheta, according to the government, used the hacker name Resili3nt, and several variations—resjames, resilient24, Resilient, ResilienT, ir Resilient.

Ancheta was never under suspicion for the Auburn attack—the university didn't report the attack and the FBI did not investigate. Anyone could have launched it. But no matter the source, like all bot attacks the raid against Auburn was swift.

It arrived through Internet Relay Chat, a worldwide network of online channels that lets people exchange text messages and meet in chat rooms, either publicly or privately. IRC is the forerunner of today's instant messaging applications and has been the source of other hack attacks.

Within seconds of penetrating the university, malicious code on the invaded PC contacted an IRC channel controlled by a hacker and downloaded a server that could receive software through the File Transfer Protocol, or FTP, which transfers data and software over the Internet. Among those files was a scanner—a software probe—to find other machines to infect.

On a command from IRC, the infected PC began scanning computers on Auburn's network, looking for other computers to infect through Microsoft's LSASS bug. It sent packets of data, requests to connect, over Port 445, which Microsoft reserves as a pathway in the operating system for networked Windows PCs to share files, printers and other resources—"like going down the street knocking on doors," Wilson says. He had already closed outside access to Port 445 on Auburn's firewall after an earlier attack on that port. But with the malicious code inside the network, the firewall was helpless to stop the scans. Within minutes,

47 PCs were infected.

Wilson was tipped to the attack by Auburn's open-source intrusion detection system, Snort, which picked up the flood of data traffic on Port 445 and sent an e-mail. By examining one infected PC, he could see the attack's pattern—the same malware (FTP server, remote administration software, scanner and a chat client) kept showing up in the same Windows directory on each PC. He and his team scrambled to get the Internet Protocol addresses of the infected machines, find the network switch they were connected to and disconnect them from the campus network. But the infection was spreading so quickly that they couldn't quarantine machines fast enough.

This attack also had a twist, Wilson noticed. He saw that the chat client commandeered the buddy list from the student's instant messaging program and invited those friends to click on the link, too. Whenever the code penetrated another PC, the cycle began again.

Reviewing Snort's archived logs of the attack, Wilson remembers feeling frightened. "This was about the worst attack I'd seen," he says. "This was different from a worm or a virus. It was a live channel of communication going back and forth."

As Auburn's PCs were taken over, they sent their Internet Protocol addresses back through IRC so the various botmasters running the attack would know how many and which machines they controlled.

Alerted by the IRC messages, bots belonging to other IRC channels immediately raced to add their own malware to those freshly infected PCs using FTP, Wilson says, as if playing some life-size computer game. Messages then flew over the chat system as individual hackers took credit for penetrating PCs at specific IP addresses.

Hackers swarmed and bragged "like a bunch of schoolkids on a playground," Wilson says. He stared as the university's PCs communicated with IRC channels all over the world—from Brazil to Greece and throughout the U.S.

Several hours and 7,000 messages later, the attack ended as suddenly as it began, when the last hacker typed, "#Exit."

The invasion was over. The network traffic had died down. But Wilson was left with a hostile army of bots that he now had to subdue.

What Bad Bots Do

Bots aren't always bad. Using C++, Assembler or other low-level languages that produce compact code, a programmer can create a bot to do mundane tasks online—maybe check stock quotes or compare prices at e-commerce sites. Search company Google uses its Googlebot, for example, to collect and index documents on the Web.

In the hands of hackers, however, bots make trouble.

Ancheta, who an uncle and cousin say is self-taught on computers, didn't write his own bot code from scratch. According to his plea, he modified Rxbot, a bot strain well known among hackers and available for download at several Web sites. Most botmasters, in fact, rely on pre-written code refined over time by other hackers, says Dmitri Alperovitch, a research scientist at CipherTrust.

This is akin to how the legitimate open-source community works, Alperovitch says, where many people pool knowledge to improve a product, "but [it's] not as public."

Stealing another page from the mainstream computing world, botmasters prefer modular systems, where instructions for different tasks can be plugged into or removed from bot code depending on what the user wants to do with it. "He might want to harvest CD keys or e-mail addresses, take information from the software registry or find code for doing denial-of-service," Alperovitch explains. The bot code can install other software that records keystrokes or finds these pieces of information itself, he says: "All these are pluggable modules."

To his version of Rxbot, Ancheta added instructions to seek out computers with a specific weakness, according to the plea. Rxbot can be tweaked to exploit several unpatched Windows vulnerabilities, including LSASS.

LSASS itself should be a crucial safeguard, as it was built to handle local security and authentication, so people without passwords can't log on to individual PCs. But as Microsoft revealed in an April 2004 security bulletin labeled "critical," LSASS suffers from a buffer overflow problem that, if left unpatched, opens any computer running Windows XP or Windows 2000 to hijack.

A hacker "could take complete control of an affected system, including installing programs; viewing, changing, or deleting data; or creating new accounts that have full privileges," the bulletin warned.

A buffer is a limited amount of memory allocated to a certain task. Software creates buffers to hold data the program might need later. If you can fool the program you're targeting into overflowing that region, it's possible to inject malicious programming instructions into the machine's memory. A hacker attacking LSASS can flood its buffer with hundreds of lines of nonsense text laced with real programming instructions telling the system to do what he wants. In this case, he'd want to be authenticated as a valid user.

The garbage text crashes LSASS but leaves the instructions in memory for the computer to execute like any other execution request, such as booting up or opening a file.

As recently as last November, 19 months after Microsoft put out its initial patch, the LSASS buffer overflow was the most exploited vulnerability in networks facing the outside world, according to Qualys, a security company in Redwood Shores, Calif. Qualys studies computers at 2 million IP addresses worldwide and manages security problems for customers such as DuPont, Hershey and eBay.

Stephen Toulouse, a security program manager at Microsoft, contends that the issue isn't technical error anymore—patches exist—but a human one. "This speaks more to the importance of making sure software is up to date," Toulouse says. "Criminals will look at even the oldest of vulnerabilities and try it."

Once Ancheta's bot infiltrated an exploitable computer, the code instructed the computer to connect to a private IRC channel he had created to direct his zombie computers, according to his plea. The password to the channel was embedded in the bot code. He "owned" these machines, in hacker lingo.

Typically, Ancheta would send over IRC a command code for the activity he wanted the bot to perform—open a certain port and start sending spam, or continue scanning a range of Internet Protocol addresses for PCs with particular software flaws, for example. At any given time, several dozen to several thousand bots would go to this spot looking for instructions.

Newer bot attacks are even more insidious, says Gary McGraw, chief technology officer at Cigital, a software quality consultancy in Dulles, Va., and author of the book Software Security. Bots can now come as rootkits—code that embeds itself into the operating system and can modify key functions performed by the system.

In a setup like Ancheta's, the bot program is visible, at least to a technology professional who knows where to look. It sits in an area on the operating system known as the user space, along with common applications like Web browsers and word processors.

But when a bot is coded as a rootkit, the bot inserts itself into what is called the kernel space, close to a computer's core operating system. The kernel is where behind-the-scenes programs such as network drivers communicate with the operating system or access the computer's hardware. Because a rootkit can modify key functions performed by the operating system, it can conceal the bot code.

For example, if antivirus software requests from the operating system access to a particular memory location to check it for malicious code, the rootkit can intercept the request and provide the security software with fake data saying, in essence, that everything is OK.

Bots that "can't be seen" by current antivirus software, McGraw says, can live longer on an infected system.

Worse, a highly skilled botmaster can use a rootkit to insert a bot into a computer's hardware, McGraw says. Specifically, the Erasable Programmable Read-Only Memory chip in every computer, which holds data when the power is turned off, can be violated, in a technique called "flashing the EPROM." If the computer survives this procedure, it becomes permanently infected.

Yet the simplest means of infiltrating a corporate network is still to supplement bug exploits with trickery, says a person interviewed by Baseline who claims to be SoBe, Ancheta's accomplice in Boca Raton, Fla. That is, getting people like the one at Auburn University to click on a link.

For example, according to SoBe, an employee may take his laptop home to browse the Web over a weekend. He doesn't know it, but bot code rides into his system when he downloads a freeware application for, say, tracking local weather. Also unknown to him is the fact that the botmaster then uploaded a virus that will spam an instant message to the employee's buddy list when he plugs into the corporate network on Monday. The message might read, "Hey, check out my new pictures," and give a hyperlink that, when clicked, sets off a bot.

Since the note is from a friend, many of the people receiving it will indeed click, thereby infecting themselves and growing the botnet.

Such social engineering, say security experts, is highly effective and growing. "It happens all the time with very annoying frequency," says TransUnion chief security officer Lines.

SoBe agrees. "Basically, some of these spamming methods rely on friendship," he says. "You don't use an exploit to infect people, you use their stupidity."

Can">

Catch Me If You Can

Catching a botmaster takes time, even one brazen enough to advertise his bot products on Internet Relay Chat channels and talk openly with prospective customers. In the Ancheta case, the FBI had been watching Ancheta for at least four months before the first time they raided his base of operation.

At 6 a.m. on Dec. 10, 2004, 10 FBI agents, armed with guns and a search warrant, converged on a small one-story house in Downey, Calif., where Ancheta operated. The yard was well kept, one of the nicest on the street. Sometimes a child's tea set sat in the front window. But agents always worry about what surprises they might find inside. Standard procedure for the FBI means that many agents go along to execute a warrant, says the FBI's McGuire, who was at the house that day. "We want to make sure we have sufficient numbers for those crazy enough to think that violence is a way to get out," he says.

But there was no violence when the FBI knocked on Ancheta's door. He was home with his mother and 7-year-old sister, who was getting ready for school.

By this time, Ancheta had attracted so much attention that several sources—people who quietly help the FBI watch for crime on the Internet—reported him, according to McGuire, who heads the cybercrime squad in Los Angeles.

Still, even cavalier botmasters can outrun law enforcement for several months. And that's what Ancheta had done. He changed Internet service providers, e-mail addresses, instant messaging handles, domain names and IRC channels, making it harder to piece together his activities.

In the four months it took the FBI to assemble their evidence and get their search warrant, Ancheta's armies attacked thousands more computers—and the FBI didn't even know yet about Ancheta's adware business. In just five weeks, from Nov. 1 to Dec. 7, the government estimates that Ancheta's botnets installed adware on 35,719 PCs without their owners' permission.

But over the second half of 2004, the FBI built its case. On Aug. 31, Ancheta sold 2,000 bots to an undercover FBI agent and assured him they were strong enough to conduct a synflood attack. The agent told Ancheta he wanted to "drop [the] site" of a business competitor, according to the government's indictment.

The agent talked with Ancheta over AOL Instant Messenger throughout the month of August, approaching him as three different people. With each persona, Ancheta discussed sales of bots that could send spam and conduct denial-of-service attacks. When the agent agreed to buy the bots, transferred money through PayPal, and received a file so his IRC channel could accept the bots when Ancheta directed them there, the FBI recorded a Camtasia video (a series of screen shots) of the bots rallying to the channel. They recorded Ancheta's voice when he spoke to the agent on the phone, and corroborated his identity by getting him to acknowledge online where he lived and then matching that to his subscriber information in logs from his Internet service providers.

According to McGuire, when the FBI showed up at his house, Ancheta was cooperative and polite. Agents seized his computers and other items and told him he was "way into felony territory." The FBI did not arrest him then because they wanted more evidence. But Ancheta gave agents a statement acknowledging that he sold botnets, McGuire adds, and said he would get a lawyer after the FBI presented its case to the U.S. Attorney's Office.

The agents took Ancheta's computers and other seized items back to FBI offices to examine. It was not until then that they discoved he was using botnets to install adware on captive PCs.

Around the same time, the FBI received a report from the China Lake Naval Air Weapons Station, saying it had been attacked by a botnet. China Lake did not return calls seeking comment. But McGuire says agents started comparing evidence from Ancheta's computers with information from the Naval Criminal Investigative Service, and found programs and Internet Protocol addresses that matched.

And less than a month later, the FBI's sources told the agency that Ancheta was back in business. The agents working the case were stunned. "We couldn't believe he went out and did it," McGuire says. "We told him it was illegal. We thought we had somebody who acknowledged what he'd done was wrong and stopped."

"[This case] was like having a snag in a jacket where you pull a thread and it keeps unraveling," says prosecutor James Aquilina, assistant U.S. Attorney in Los Angeles.

McGuire believes Ancheta resumed business because he couldn't walk away from the money. He'd bought a BMW, a 1993 325is, for which he claimed to pay $6,000 in cash, according to postings on Bimmerwerkz.com, a forum for BMW enthusiasts where he sought advice and posted pictures of his car. He also talked about muscling up his car with chrome rims, tail lights and other special accessories that one BMW expert estimates cost $7,000.

In any event, Jan. 9, 2005, was a busy day for the FBI. Computers at Northwest Hospital in Seattle were attacked by a botnet, and although it was not one of Ancheta's, it exploited the same Windows LSASS bug that had been used against Auburn four months earlier, according to an affidavit filed in federal court in Seattle against 20-year-old Christopher Maxwell. He's pleaded not guilty. His lawyer, Steven Bauer, did not return calls seeking comment.

On that same day, Ancheta's botnets were found invading computers at the Defense Information Systems Agency in Arlington, Va. "Multiple machines in their network were infected, called out to IRC and were directed to pick up adware or for other nefarious purposes," Aquilina says.

But the FBI now had a laser focus on botnets, and Ancheta's days as a free man were numbered. As he and SoBe managed their bots to avoid attention from the FBI, the adware vendors and other botmasters—redirecting bots among different IRC channels, according to the plea agreement, so they weren't always in one place, or varying how fast they downloaded adware to imitate normal Internet traffic—the bureau prepared for more search warrants.

Over the next 4 1/2 months, the FBI again analyzed evidence. For example, when Special Agent Cameron Malin reverse-engineered code from one of Ancheta's bots, he found an affiliate ID number from GammaCash, an adware company in Quebec. It matched the account number on a pay stub and a check made out to Ancheta from GammaCash, for $2,352.66, that agents had taken from his mother's house. Every time one of Ancheta's botnets infected a PC and installed adware, GammaCash was crediting Ancheta's account. GammaCash did not return calls seeking comment.

On Thursday, May 26, 2005, at 6 a.m. Pacific time, teams of FBI agents showed up on Ancheta's mother's doorstep in Downey one more time, and at SoBe's house in Boca Raton, and at Sago Networks, an ISP headquartered in Tampa whose servers the government says were the source of Ancheta's attacks on the Department of Defense—all at the same moment. They seized computers at all three places. Sago declined to comment about the case other than to say that it fully cooperates with law enforcement.

With that second raid, the FBI believed Ancheta and SoBe were out of the bot game, McGuire says. Their computers, ISP and server hosting infrastructure were unavailable, and their bots had nowhere to rally, although the checks from the adware companies rolled in until August, according to Aquilina.

Ancheta, however, was not arrested until Nov. 3. Aquilina and Malin were suddenly asked to help with another high priority case, an investigation in which lives were threatened. And the government had to review evidence, study the offending code, line up experts to testify and prepare for prosecution. "We did not want to make a false step," McGuire says. "We had to make sure we had all our ducks in a row."

On the morning of Jan. 23, 2006, Ancheta was led into a federal courtroom in Los Angeles. He wore the baggy green outfit of an inmate, because he has been held at a maximum-security detention center downtown since his arrest in November. With family in the Philippines, Ancheta was considered a flight risk. The average age of prisoners at the facility is 37, and their crimes include drug trafficking and murder. But prosecutor Aquilina says he and Ancheta's lawyer, federal public defender Greg Wesley, agreed to put Ancheta in the detention center instead of the county jail, partly to ensure his safety and partly because Wesley needed him nearby so they could discuss the complex case.

Ancheta pleaded guilty to two counts of fraud and two counts of computer crime, including attacking China Lake. He faces up to 25 years in prison.

Neither the U.S. Attorney's Office nor the FBI will comment on SoBe.

In the courtroom, according to a transcript, Judge R. Gary Klausner asked Ancheta whether he understood each charge and what he would give up if he pleaded guilty—not just the right to a trial, but the right to vote, serve on a jury, own a gun.

"Yes, Your Honor," he responded. "Totally."

Ancheta was returned to the prison to wait for his May sentencing. Only his lawyer and immediate family may visit.

The Fallout

The government continues to investigate leads related to Ancheta case. As Resili3nt and other online aliases, Ancheta admitted conducting more than 30 transactions with 10 people, all unindicted co-conspirators. Indeed, Ancheta is the type of criminal the FBI expects to see more of, McGuire says: "One of those profit-motivated individuals ... who exploited the latest in tech."

Companies fighting bot attacks shouldn't feel secure just because the government has nailed a few botmasters. Like law enforcers, corporate and government technology managers are struggling to stay ahead of bot crimes. Despite Microsoft's monthly Patch Tuesday—a day that has been marked by a steady stream of security patches issuing from Redmond—companies still take an average of 19 days, or almost three weeks, to fix critical vulnerabilities on just half of the externally facing systems on their networks, according to security vendor Qualys. Fixing half of their internally facing systems takes more than twice as long—nearly seven weeks. Botmasters have lots of time to exploit them.

At Auburn University, stamping out the infection took about three weeks. That's because some PCs were cleaned with antivirus software that didn't remove the bot code, so they reinfected the network once they were allowed back on. Ultimately, all of the PCs' hard drives had to be wiped clean by Wilson's staff. "That's the only foolproof way to get this crap off," he says.

Eighteen months later, Auburn is still educating students, faculty and administrators not to click on links or open attachments in e-mails or instant messages. Students are required to watch a streaming video on cybersecurity when they sign up for Internet services. They must also enter the university's network through a Cisco portal that checks their PCs to see if their antivirus software and the patches on their Microsoft software are up to date. If not, their PCs are updated on the spot. Auburn also blocked inbound and outbound access to Internet Relay Chat on the university's firewall.

Auburn has had no more major bot attacks, Wilson says. But that's no guarantee that Auburn will remain bot-free. Different colleges within the university have different rules on what users can do with computers, although discussions on how to enforce computer security have begun. "We're not corporate," he says. "We'll take computing rights away, but the school of engineering may be stricter than arts and sciences."

Likewise, corporations probably will never be fully inoculated against bots, according to Kris Palmer, chief information security officer at The Mosaic Co., a

$4.5 billion agriculture company in Plymouth, Minn. "There are so many points of insecurity that you have to pick your battles where you are weakest," she says.

Every company should run as much security technology as it can at each level of computing—desktop, server, internal network and external Internet connections, advises TransUnion's Lines. That includes firewalls, antivirus software, automated patching programs, intrusion detection systems, e-mail protection gateways and anti-adware applications, he says.

More specific steps include closing ports that aren't used in particular applications. For example, consider closing ports 6666 and 6667, which communicate with Internet Relay Chat, as Auburn did. Microsoft also recommends blocking certain ports at the firewall level, including ports 135, 137, 138 and 139, which allow applications on different computers to communicate; port 593, which allows computers to talk to each other over the Web; and port 445, an entry point for some worms and bots such as Sasser, Agobot and Zotob and the vehicle for spreading the infection at Auburn. In addition, block all unsolicited inbound traffic on ports with numbers higher than 1024, Microsoft advises.

Also, the experts say, understand the typical ebb and flow of traffic on the corporate network so that you'll recognize unusual patterns early. Look at network logs regularly, McGraw, the software quality expert, says. "Wonder why your machine is doing stuff when you're not actually using it," he says.

Corporate network administrators should learn how to disrupt a botnet attack, Palmer advises. Isolate an infected machine from the internal network, as Wilson did at Auburn, then study the bot code inside it. Identify the vulnerability it used to get into your system and fix it. Palmer plans to put her technology staff through "ethical hacking" training so that they can know the enemy, she says.

"Not that we want to teach anyone to hack, but knowing what it is to hack. What it looks like. What to look for," she says.

But despite all efforts, Lines, for one, doesn't think botnets will be eradicated, merely mitigated. "The preventions you have in place today won't prevent the attacks of tomorrow," he points out. "It's an arms race."