Error budgets for teams that have never had one

Most teams adopting error budgets start by writing forty of them. Three months later nobody looks at the dashboard, because forty numbers is not a signal, it is wallpaper.

Start with one. One user journey, one indicator, one target, and one written rule about what changes when the budget is gone. That is a working error budget. The other thirty-nine can wait until the first one has changed a decision.

Pick the journey the business would phone you about

Not "the API". Not "the cluster". A journey a named person outside engineering cares about: a driver scans a parcel and gets a confirmed handover, or a patient record loads for a clinician in under two seconds.

At Vanta Logistics the first SLO we wrote was for dispatch acceptance — a depot submits a manifest and receives an accepted or rejected response. One endpoint, one meaning, one owner. Availability of the wider platform was already being measured and had never once predicted an angry phone call.

The target is a budget, and the budget is smaller than it sounds

Availability targets are easier to reason about as permitted downtime. Over a 30-day window:

Show this table to the person asking for "four nines" and the conversation changes shape immediately. Four nines means a single bad deploy noticed and rolled back in four minutes, every month, forever, including the month your platform lead is on leave. Almost nobody wants to buy that. They want to say it.

Our advice for a first SLO is 99.5% or 99.9%, chosen by looking at the last 90 days of real data. If your current performance is 99.2%, do not set 99.9% — you will be out of budget before the quarter's first sprint review and everyone will learn that the number is theatre.

Measure it where the user is

Count events at the edge, as a ratio of good events to valid events. Do not derive availability from host uptime, and do not average latency; average latency is a number that no user has ever experienced.

# Multi-window, multi-burn-rate alerting for a 99.9% target.
# Fast burn: 2% of a 30-day budget in 1 hour  → factor 14.4 → page a human.
# Slow burn: 5% of a 30-day budget in 6 hours → factor 6    → raise a ticket.
#
# Both windows must agree, so one bad scrape cannot wake anyone up.
groups:
  - name: slo-dispatch-acceptance
    rules:
      - alert: DispatchAcceptanceFastBurn
        expr: |
          job:slo_bad_ratio:rate1h{journey="dispatch-accept"} > (14.4 * 0.001)
          and
          job:slo_bad_ratio:rate5m{journey="dispatch-accept"} > (14.4 * 0.001)
        for: 2m
        labels:
          severity: page
      - alert: DispatchAcceptanceSlowBurn
        expr: |
          job:slo_bad_ratio:rate6h{journey="dispatch-accept"} > (6 * 0.001)
          and
          job:slo_bad_ratio:rate30m{journey="dispatch-accept"} > (6 * 0.001)
        for: 15m
        labels:
          severity: ticket

Burn rate is what makes an error budget operational. A raw threshold alert tells you something is wrong now; a burn-rate alert tells you whether, at the current rate, you will still have a budget on the last day of the month. That is the question worth waking someone for.

Write the policy before you need it

An error budget with no consequence is a chart. The consequence does not need to be dramatic, but it must be automatic and written down in advance, because negotiating it during an outage guarantees it is negotiated in favour of shipping.

The policy we most often land on is three lines:

1. Budget remaining: feature work proceeds; risky changes go out behind flags on Tuesdays and Wednesdays.
2. Budget exhausted: production changes are limited to reliability work and rollbacks until the trailing 30-day window recovers. Exceptions require the accountable director's name in the change record.
3. Budget untouched for two consecutive months: the target is too loose. Raise it, or spend the surplus deliberately on a migration or a chaos exercise.

Line three is the one teams skip, and it is the one that keeps the system honest. An error budget that is never spent is not evidence of excellence, it is evidence that you are over-engineering somewhere and could be moving faster.

What the first quarter looks like

One journey instrumented. Two alerts. One policy page. One monthly review, twenty minutes long, that answers a single question: did the budget change any decision this month? If the answer is no for three months running, the target is wrong or the journey is wrong. Fix that before you write the second SLO.