Performance and load testing
Reading results — response times, percentiles, errors
A load test produces a lot of numbers. Which ones matter, why the average lies and p95 doesn't, what throughput and error rate tell you, and how to read the charts.
After a load test you have thousands of measurements. A handful of numbers summarise them — if you know how to read them.
Response time (latency)
Response time — also latency — is how long one request took: from sending it to receiving the whole answer, in milliseconds (ms; 1,000 ms = 1 second). For people, as a rough guide:
| Time | Feels |
|---|---|
| under 100 ms | instant |
| 100–300 ms | quick |
| 300 ms – 1 s | noticeable |
| 1–3 s | slow; attention wanders |
| over 3 s | many people give up |
An API call is usually one of several behind a screen, so APIs need to be faster than the screen as a whole.
Why the average lies
Imagine 100 requests: 95 take 50 ms, and 5 take 5 seconds — a database hiccup. The average is (95 × 50 + 5 × 5000) ÷ 100 = 297 ms. It describes nobody: 95 people got 50 ms and 5 people waited 5 seconds. And with a busy site, "5 in 100" is thousands of unhappy people a day.
So we use percentiles: line all the response times up from fastest to slowest, and read the value at a position.
- p50 (the median) — the middle: half were faster. The typical experience.
- p95 — 95% were faster than this; 5% were slower. The experience of the unlucky few — the number most goals use.
- p99 — 99% were faster; 1 in 100 was slower. The tail — where problems hide.
- max — the single slowest. Often a fluke; worth a look, not worth a goal.
In the example: p50 = 50 ms, p95 = 50 ms… and p99 = 5,000 ms. The percentiles tell the true story; the average hid it.
Throughput: requests per second
Throughput — requests per second (RPS) — is how many answers the API gave each second. On its own it says little ("we did 400 RPS" — with what response times? how many failed?). It matters in relation to the others:
- throughput rises with load while response times stay flat → the API has room;
- throughput stops rising while response times climb → the API is at its limit (more load only makes queues longer);
- throughput falls while load rises → the API is overloaded and getting worse.
Error rate
The error rate is the share of requests that failed: no answer at all (a timeout, a refused connection), or a status of 400 and above. And — if your requests have checks — answers that came back but were wrong.
- Any 5xx under load means something gave way. Find the first one in time: that's roughly where the limit is.
- 429 or 403 in bulk usually means a rate limit or firewall — the numbers then describe the guard, not the API (testing safely).
- Timeouts mean answers took longer than the tool would wait.
Reading the charts
Most reports draw these over time. The patterns to recognise:
| You see | It usually means |
|---|---|
| Response times flat while users rise | Plenty of room |
| Response times rise with the users, then flatten when users do | Normal queuing — check whether it's still within the goal |
| Response times rise steadily while load is constant | Something grows over time — a leak, a queue, a filling table (soak tests find this) |
| Requests per second hit a ceiling; response times climb | The capacity limit |
| Errors start at a clear point | The breaking point — note the users / RPS at that moment |
| Sawtooth: slow, fast, slow, fast | Periodic pauses — garbage collection, a scheduled job, autoscaling |
| Slow only at the very start | Warm-up: cold caches, connections being made — ignore the first minute, or ramp up |
Other numbers you'll see
- Time to first byte (TTFB) — from sending until the first byte of the answer: mostly the server working. If TTFB is most of the response time, the time is spent in the API, not the network.
- Connection and TLS time — making the connection and the encryption; high when connections aren't reused.
- Apdex — a 0-to-1 score of how many answers were "satisfying" (within a target) or "tolerable" (within 4× it).
- Users / VUs — how many simulated users were running at each moment.
Key takeaways
- Response time per request; summarise with percentiles, not the average: p50 typical, p95 the unlucky few, p99 the tail.
- Throughput matters with the others: flat while times climb = the limit. Error rate first — failures are fast and flatter the speed.
- Learn the chart shapes: rising under constant load = something grows; a ceiling in RPS = capacity.
In the app: Reading the report · Key numbers · The result
Next: Finding the bottleneck