Performance and load testing

Reading results — response times, percentiles, errors

A load test produces a lot of numbers. Which ones matter, why the average lies and p95 doesn't, what throughput and error rate tell you, and how to read the charts.

After a load test you have thousands of measurements. A handful of numbers summarise them — if you know how to read them.

Response time (latency)

Response time — also latency — is how long one request took: from sending it to receiving the whole answer, in milliseconds (ms; 1,000 ms = 1 second). For people, as a rough guide:

TimeFeels
under 100 msinstant
100–300 msquick
300 ms – 1 snoticeable
1–3 sslow; attention wanders
over 3 smany people give up

An API call is usually one of several behind a screen, so APIs need to be faster than the screen as a whole.

Why the average lies

Imagine 100 requests: 95 take 50 ms, and 5 take 5 seconds — a database hiccup. The average is (95 × 50 + 5 × 5000) ÷ 100 = 297 ms. It describes nobody: 95 people got 50 ms and 5 people waited 5 seconds. And with a busy site, "5 in 100" is thousands of unhappy people a day.

So we use percentiles: line all the response times up from fastest to slowest, and read the value at a position.

  • p50 (the median) — the middle: half were faster. The typical experience.
  • p95 — 95% were faster than this; 5% were slower. The experience of the unlucky few — the number most goals use.
  • p99 — 99% were faster; 1 in 100 was slower. The tail — where problems hide.
  • max — the single slowest. Often a fluke; worth a look, not worth a goal.

In the example: p50 = 50 ms, p95 = 50 ms… and p99 = 5,000 ms. The percentiles tell the true story; the average hid it.

Throughput: requests per second

Throughput — requests per second (RPS) — is how many answers the API gave each second. On its own it says little ("we did 400 RPS" — with what response times? how many failed?). It matters in relation to the others:

  • throughput rises with load while response times stay flat → the API has room;
  • throughput stops rising while response times climb → the API is at its limit (more load only makes queues longer);
  • throughput falls while load rises → the API is overloaded and getting worse.

Error rate

The error rate is the share of requests that failed: no answer at all (a timeout, a refused connection), or a status of 400 and above. And — if your requests have checks — answers that came back but were wrong.

  • Any 5xx under load means something gave way. Find the first one in time: that's roughly where the limit is.
  • 429 or 403 in bulk usually means a rate limit or firewall — the numbers then describe the guard, not the API (testing safely).
  • Timeouts mean answers took longer than the tool would wait.

Reading the charts

Most reports draw these over time. The patterns to recognise:

You seeIt usually means
Response times flat while users risePlenty of room
Response times rise with the users, then flatten when users doNormal queuing — check whether it's still within the goal
Response times rise steadily while load is constantSomething grows over time — a leak, a queue, a filling table (soak tests find this)
Requests per second hit a ceiling; response times climbThe capacity limit
Errors start at a clear pointThe breaking point — note the users / RPS at that moment
Sawtooth: slow, fast, slow, fastPeriodic pauses — garbage collection, a scheduled job, autoscaling
Slow only at the very startWarm-up: cold caches, connections being made — ignore the first minute, or ramp up

Other numbers you'll see

  • Time to first byte (TTFB) — from sending until the first byte of the answer: mostly the server working. If TTFB is most of the response time, the time is spent in the API, not the network.
  • Connection and TLS time — making the connection and the encryption; high when connections aren't reused.
  • Apdex — a 0-to-1 score of how many answers were "satisfying" (within a target) or "tolerable" (within 4× it).
  • Users / VUs — how many simulated users were running at each moment.

Key takeaways

  • Response time per request; summarise with percentiles, not the average: p50 typical, p95 the unlucky few, p99 the tail.
  • Throughput matters with the others: flat while times climb = the limit. Error rate first — failures are fast and flatter the speed.
  • Learn the chart shapes: rising under constant load = something grows; a ceiling in RPS = capacity.

In the app: Reading the report · Key numbers · The result

Next: Finding the bottleneck

A question these lessons don’t answer? Write to support@apitestcenter.com — we add lessons for the questions people ask.