How Good Are Your Service Level Agreements?

One of the most important steps when contracting an IT service provider is setting the performance metrics of the service-level agreements (SLAs).

Unfortunately, many organizations treat this step as an afterthought, says Ian Hayes of Clarity Consulting, a California-based provider of training and development services.

As an IT management consultant with expertise in vendor negotiation and contract development, Hayes believes that organizations could get more out of their service providers if they put more thought into their SLAs.

These agreements are more than just setting terms of service and expectations; they define the relationship between service providers and their clients, as well as performance measurements and penalties for performance breaches.

The following are Hayes’ five best practices for effective SLAs.

  • Establish Business Metrics
  • Establish Operational Metrics
  • Hold Service Providers Accountable
  • Exercise Penalties and Incentives
  • Measure Third-Party Experience

    Establish Business Metrics
    Before you do anything else, define business objectives for obtaining the service in the first place and establish metrics to measure how well the service is meeting these business objectives.

    “If I’m the business person, all I really care about is that the service is working the way I want it to work,” Hayes says. “I don’t really care about the details of whether it was 99.7 uptime or 99.9. I only care whether I got it when I wanted it.”

    An example: A medium-size, non-global organization is bringing in a provider to manage e-mail to save money and provide a higher level of service than what it can offer with its in-house resources. Important business metrics in this case would be how much the service costs per message, and whether users are able to access messages the way they want them. Because the firm is not global, paying for perfect availability around the clock may not be necessary. It might just be important to look at how well the provider is at maintaining availability from 6 a.m. to 8 p.m.

    Establish Operations Metrics
    Once you’ve set business metrics, also set operations metrics to aid the IT organization’s monitoring of day-to-day service performance and identify performance problems and lapses. Operational metrics should include theses categories: volume, responsiveness, quality and efficiency.

    “Any one of these metrics in and of themselves doesn’t tell you enough,” Hayes says. “The example that I like to use is say you are driving in your car and you have a dashboard with all of these measurements. If the only thing you look at is speed, that is well and good because it keeps you from getting tickets. But if you go on a long trip and you’ll run out of gas. Really what you need is a whole set of these metrics and it will give you a complete picture of how the car is running.”

    An example: An organization is contracting Tier 1 call center support. A solid operations metrics is the number of calls per month (volume), how quickly support staff answered calls (responsiveness), how well the caller’s issues were resolved (quality) and cost per call and resolution.

    Hold Service Providers Accountable
    If you’re going to measure a service providers’ performance, you need have controls, procedures and penalties for when service falls below contractual requirements. Many SLA agreements set parameters for how quickly a provide must respond to remedy a performance lapse or break in availability—this is particularly important for managed services agreement. The SLA should provide discounts, fines and punitive penalties if the provider fails to correct the problem within prescribe limits.

    SLA accountability requirements and accountability go both ways. A contracting company must deliver on its requirements to deliver access to infrastructure and information and materials so the service provider can meet its expectations.

    “Make sure the metrics you are holding the vendor accountable for are metrics that are within the control of the vendor,” Hayes says. “The best way to do this is through two-way service level agreements. In the two-way service level agreement for development, for example, the vendor would say ‘I will deliver you the software on the date agreed upon if you get me the specs by the following dates and it is to the following level of completeness.’ If the client then fails their side of the agreement then the vendor isn’t held accountable for that.

    An example: A company contracts a software as a service (SaaS) provider for a Web-based application that many of its end users need to access. Over the course of several months, users complain of difficulties in connecting to the service. It comes to light that the availability and access issues stems from networking problems on the client’s end. A good set of SLA metrics will not penalize the provider for issues with availability, as they had no control over their client’s equipment.

    Exercise Penalties and Incentives
    Metrics and provides companies with some insurance against poor service. However, IT and project managers should exercise restraint and rational thinking when applying penalties. In some cases, IT managers may consider providing service providers with incentives rather than fining them for poor performance.

    “Whenever putting a penalty or incentive in a contract you really have to think through every angle,” Hayes said. “What possible side affects could this cause that you didn’t mean to cause?”

    An example: An organization contracts with a development firm to create a custom application. In it is a clause offering a bonus for delivering the package one month early. The developer hits the early target date, but the application is riddled with bugs. The unintended consequence of this incentive was a rush to finish that enticed the vendor to skimp on testing. Ideally the organization should have specified quality metrics within the incentive to prevent the problems.

    Measure Third-Party Experience
    The ultimate measure of service success is the user experience. Standard SLA metrics mean nothing if the users—outside clients and customers or in-house employees—find no value in the applications and services provided. Companies should survey users for their satisfaction and make adjustments based on their ratings, experiences and suggestions.

    “If your vendor meets the metrics and you do a customer satisfaction but end users are really unhappy then you must not be measuring the things they care about,” Hayes says.

    An example: An organization contracts with a Tier 1 support call center and emphasizes efficiency and responsiveness metrics. The vendor is doing very well at answering calls quickly and cheaply. A customer satisfaction survey is issued and the results are awful—customers are unhappy because they are being rushed off the phone without proper resolution. The organization likely needs to tweak metrics for quality of service measurements.

    SOUND OFF: How are you measuring and enforcing SLA agreements? Share your experience with Baseline at [email protected].