Performance and load testing

Finding the bottleneck

When an API slows down under load, something specific ran out first — CPU, a connection pool, a database lock, a slow dependency. The usual suspects, the signs each one leaves, and how to narrow it down.

A load test tells you that an API slows down at 200 users. The next question is why — and the answer is almost always that one thing ran out first. That thing is the bottleneck: the narrowest part of the bottle, which decides how fast everything flows. Make anything else faster and nothing changes; widen the bottleneck and the whole system speeds up — until the next-narrowest part becomes the bottleneck.

Queues: why "a bit slower" becomes "suddenly unusable"

Everything that can only do so much at once — a CPU, a pool of database connections, a single-threaded worker — has a queue in front of it. While requests arrive slower than it can handle them, the queue stays empty and response times stay flat. As arrivals get close to its limit, the queue grows faster and faster: at 80% busy, a request might wait as long as it takes to process; at 95%, many times longer. That's why response times look fine, fine, fine — and then shoot up over a small increase in load. The point where they bend is the capacity of the bottleneck.

The usual suspects

BottleneckSigns under loadCommon causes
CPUServer CPU near 100%; times rise steadily with loadHeavy computation, inefficient code, JSON of huge answers, encryption, too few servers
MemoryMemory climbs over time; pauses; eventually crashes or restartsLeaks, caches with no limit, loading whole tables into memory
DatabaseDB CPU or I/O high; specific requests slow; locks and timeoutsMissing indexes, slow queries, too many queries per request (N+1), lock contention on hot rows
Connection poolTimes jump at a fixed concurrency; "timeout waiting for connection" errors; DB itself not busyPool too small for the load; connections held too long (slow queries, forgotten closes)
Threads / workersA fixed number of requests in progress, the rest wait; CPU not maxedToo few worker threads; blocking calls (waiting on I/O) in a small pool
A slow dependencyOne external call dominates the time; your servers idleA downstream API, a payment provider, an auth server — slow, rate limited, or failing
LocksTimes grow with concurrency even at low CPU; deadlocksEveryone updating the same row, counter or file
Network / bandwidthBig answers slow; time in transfer, not in the server (low TTFB, high download)Huge payloads, no compression, the test machine's own network
Rate limits / firewallsSudden 429s or 403s at a round numberA gateway, CDN or WAF protecting the API — by design
The load generatorEvery request slows at once; the test machine's CPU is maxed; requests not sentThe machine running the test can't keep up

The N+1 query: the most common one

An API returns a list of 50 orders. The code runs 1 query for the orders, then 1 more per order for its customer — 51 queries. Each is fast; together, they're slow, and they multiply under load. The fix (one query with a join, or a batch fetch) often makes an endpoint 10× faster. Signs: response time grows with the size of the answer, and the database sees many tiny identical queries.

How to narrow it down

  1. Find where the time goes in one request. Is it the network (connection, TLS, download) or the server (time to first byte)? A high TTFB means inside the API.
  2. Test endpoints separately. If /products stays fast and /orders slows, the problem is in what /orders does — not the servers in general.
  3. Change one thing at a time and compare: more users, a bigger page size, a different endpoint. Two changes at once tell you nothing.
  4. Watch the server while you test. A load test only sees the outside. Look at the API's own monitoring at the same time: CPU, memory, the database's busiest queries, the connection pool, garbage-collection pauses, errors in the logs. The load test makes the problem happen; monitoring shows where.
  5. Find the knee. Run a step test (rising load in steps) and note where response times bend upward or errors start. That number is your current capacity.
  6. Rule out the test itself. If the load machine's CPU is maxed or requests weren't sent, the bottleneck may be the test. Run from a bigger machine, or closer to the API.
  7. Fix the bottleneck, then test again — the next one is now in charge.

Things that fool you

  • Warm-up. The first minute is slower: caches fill, connections open, code gets optimised (JIT). Ramp up, and judge the steady part.
  • Caching. The same request a thousand times may be served from a cache — fast and meaningless. Use test data so requests differ like real ones.
  • Autoscaling. A cloud system may add servers mid-test: times rise, then fall. Know whether it's on, and how long it takes to react.
  • Shared environments. Someone else's test on the same system makes your numbers worse. Book the environment.
  • A different environment. A test server with a quarter of production's resources finds a quarter of its capacity — useful for comparing, risky for promising.

Key takeaways

  • One resource runs out first — the bottleneck — and queues in front of it make response times bend sharply near its limit.
  • The usual suspects: CPU, memory, database (and N+1 queries), connection pools, threads, slow dependencies, locks, rate limits — and the load machine itself.
  • Narrow it down: time in the server or the network, one endpoint and one change at a time, and watch the server's monitoring during the test.

In the app: Reading the report · Compare

Next: Goals and regressions

A question these lessons don’t answer? Write to support@apitestcenter.com — we add lessons for the questions people ask.