Scrutinizing AssessorsBy Ericka Chickowski | Posted 2008-03-28 Email Print
As details trickle out about
Most security experts will tell you that there is no such thing as a perfect security playbook. The dynamic nature of IT security calls for different 'plays' that suit each team - which is why most security standards must contain some level of vagueness for customization. Though it is widely regarded as the most thorough and specific of all security standards, the PCI rule set is still open to interpretation. And the folks who must shoulder that responsibility of interpreting are the assessors performing audits prior to rubberstamping organizations as “PCI Compliant.”
There are currently more than 120 QSAs authorized by the PCI Security Council to conduct PCI compliance audits and each has their own interpretation of PCI.
“It comes down to interpretation,” said Chris Konrad, senior vice president at Fortrex Technologies, a QSA. “You know, there's still room for interpretation with the audits and since each of these assessors may interpret the standards or requirements differently it’s possible that they're not doing the due diligence they need to do."
For example, Pinkett of Core Security says that when it comes to penetration testing, all PCI mandates is that some form of penetration testing is executed. But it doesn’t delineate what ‘penetration testing’ really means for the purpose of certification.
“So that could mean somebody looked at one system to see if there were vulnerabilities or that could mean that a team of one hundred tried to break in with every possible tool in the world,” Pinkett said. “So it comes down to at the end of the day, the responsibility of the company getting the assessment, the responsibility of the assessor to be well-educated and perform at the best-practices level."
*Baseline editor-in-chief Lawrence Walsh writes Hannaford Bros. breach proves, yet again, that there's no such thing as "unbreakable" security.
Hannaford has chosen to stay mum on which company performed its PCI compliance assessment and it will be a long time before it becomes clear whether this assessor did its job poorly. But some security experts believe that as legal wrangling over the breach heating up, and Hannaford may very well publicly roll over on its assessor if it really was compliant during the breach.
“My guess is that the assessor here is probably very nervous,” Litan said.
The situation could turn into an ugly bout of finger pointing, Kelley agreed.
More importantly, though, this case of a compliant company suffering a breach may spark discussion about the governance of the PCI Qualified Security Assessors program.
“I think this issue is going to flare up, because the assessors do have different interpretations of the standard. Nobody will step in and actually say 'This is the bottom line' because the council is in charge of the standard but not compliance,” Kelley said. “So this could be a sort of a flash point for focus on the assessors and how the council's managing them, because there have been complaints about quality of assessors and some assessors trying to sell products as part of the compliance process."
As the system currently stands, merchants choose which assessors they work with and some are definitely more stingy than others with PCI certifications. Many organizations realize this and some may go about choosing an assessor like the lazy college kid chooses a professor, pick the prof that gives the highest grades, Pinkett says.
The difficulty, Rothman says, is that there is no way to completely eliminate the human element of assessment. “It all gets back to the opinion of the assessor, and there is no way to institutionalize that,” he said. “I mean you can certainly try to push for standards, you can make everything into a sort of cookbook but at the end of the day you're still going to need a person that’s assessing and providing their opinion on what another set of people are doing."