Performance and load testing

Goals and regressions

"Fast enough" needs a number. How to set performance goals (SLOs) people can agree on, turn them into pass/fail targets, and catch an API getting slower from one release to the next.

A load test without a goal ends with a shrug: "p95 was 340 ms… is that good?" A goal turns the result into an answer.

SLIs, SLOs and SLAs

Three related words you'll hear:

  • An SLI (service level indicator) is what you measure: p95 response time, error rate, availability.
  • An SLO (service level objective) is the target for it: "p95 under 300 ms; fewer than 0.1% errors". It's the team's own promise.
  • An SLA (service level agreement) is a contract with a customer, usually looser than the SLO, with consequences (refunds) if it's broken.

For load testing, what you need is SLOs — goals specific enough to pass or fail.

Writing a good goal

A useful performance goal says what, how fast, for how many, and under what load:

  • Use percentiles, not averages (reading results) — p95 for "almost everyone", p99 for the tail.
  • Include errors. A fast API that fails 3% of the time hasn't met any reasonable goal.
  • Tie it to a load. "Under 300 ms" at 5 users and at 500 users are different promises.
  • Different endpoints, different goals. A search can take longer than a product page; a report export longer than both. One number for everything fits nothing.
  • Start from the person. How long will the user wait on that screen? How many API calls are behind it? Work backwards to each call's share.
  • Start from reality, then improve. If today's p95 is 900 ms, a 100 ms goal will just fail forever. Write down today's numbers, agree on a target, move towards it.

From goals to pass/fail

Load-test tools turn goals into thresholds: limits checked during and after the run. Breaking one fails the test — and in a pipeline, fails the build (API tests in a pipeline). Some tools can stop early when a goal is clearly missed, which saves time and stops hammering a system that's already struggling.

Remember the checks too: a goal like "at least 99.9% of answers pass their checks" makes sure speed isn't bought with wrong answers.

Regressions: getting slower without anyone noticing

A performance regression is when a change makes the API slower — a new feature adds a query, a library update is less efficient, a config change shrinks a pool. Each one is small; nobody notices; six months later the API is twice as slow.

The defence is comparing against a baseline:

  1. Run a load test you trust — same request, same load, same environment — and keep it as the baseline (a reference run).
  2. After each change, run the same test again and compare: p95, p99, errors, throughput.
  3. Decide in advance what counts as worse — "more than 10% slower at p95" — so the comparison is a decision, not a debate.

A few cautions:

  • Compare like with like — same load, same environment, same data, similar time of day. Otherwise you're comparing the conditions, not the code.
  • Expect noise. Two identical runs differ by a few percent. Small differences mean nothing; look for clear changes, or repeat the run.
  • Look at the trend, not just the last pair: a slow creep over ten releases is invisible between any two of them.

Key takeaways

  • SLI = what you measure, SLO = your target, SLA = the contract. Load tests need SLOs.
  • A good goal: percentiles + errors + a load, per endpoint, starting from today's reality.
  • Catch regressions by running the same test against a baseline after every change, with a limit decided in advance.

In the app: Goals (pass or fail) · Reference runs · Command line and CI

Next: Testing safely and realistically

A question these lessons don’t answer? Write to support@apitestcenter.com — we add lessons for the questions people ask.