Quashing a Bug Before It AlightsBy Sean Gallagher | Posted 2003-09-10 Email Print
Microsoft patching won't go away. What will you do about it?Security is again blowing a hole into Microsoft's relationships with its customersboth individual and corporate.
But even for companies unscathed by this latest round of maleficent software, Microsoft's security holes are having a significant business impact. For some companies, the prescribed cure to Microsoft software bugs may be just as bad as the disease.
"Patch management" is a euphemism for unnecessary pain. Many customers just don't have the resources to devote to testing the impact of every new patch on their existing applicationsand those that do frequently find that patches break software that they depend on to run their business.
Deploying even a single patch in panic mode can be costly. Citigroup, for example, had dozens of technical employees at each of its business units this summer working almost exclusively on deploying the latest bug fixes for more than a week, according to staff working on the problem. And this was before the Blaster burst into general awareness in August.
Citi won't comment officially, except to note it didn't suffer any security breaches. But the logistics of applying collections of patches to every single desktop computer and file server in the company's inventoryand at least four different versions of the Windows operating system across all of themis a gargantuan challenge.
According to one Citi network technician, patching was slowed down by differences in the distribution of Microsoft's service packs for Windows 2000 across the network. The patch for Windows 2000 required that Service Pack 3 for that operating system be installed, for instance. On its end, Citi lacked a consistent way to test whether patches had been applied successfully. That's a problem with installing patches on remote servers and desktops.
There were other stumbling blocks. "I had some [systems] I couldn't log onto [or] didn't have administrative rights to," Citi's technician told me. Rather than distributing the patches electronically, Citi's technical "ground-pounders" had to go out on foot and get physical access to desktop computers.
Citi isn't alone in such struggles. Bill Anderson, lead product manager for Microsoft's enterprise management division, says these sorts of problems are common to many of its customers. "Citi is probably pretty typical for a large enterprise customer,'' he says. Large enterprises "often don't have a centralized top-down approach for things like patch management, or security in general."
Microsoft's partial solution to software update woes is the Software Update Services "feature pack." This is a set of tools for its System Management Server (SMS), which package updates and automatically deploys them to systems that need them. The functionality will be an integrated part of the next version of SMS, which should be commercially available this fall.
But patching en masseeven automated patching isn't always the best answer. "You may look at [a new security hole] and say, 'I can block these ports and not have to patch right away,'" says Anderson. The only way to know what course is best is by having a good handle on what you have installedand good documentation of how your applications work.
Unfortunately, keeping your own house in order doesn't guarantee the next Microsoft security loophole won't affect you.
A big chunk of the downtime at companies hit by Blaster and its ilk was caused by computers owned by consumers. That's a problem that can only be addressed by Microsoftand corporate customers should hold Microsoft's feet to the fire to do so.