Testing APIs

Checks — deciding what a correct answer is

A check (or assertion) is a rule a correct answer must follow. Which checks to write, how strict to be, and how scripts go further when a rule needs code.

A check — in many tools called an assertion — is a rule a correct answer must follow: the status is 201, total is more than 0, the answer arrives within 500 ms. A test is a request plus its checks. Without checks, a tool can only tell you the request got an answer; with them, it tells you the answer was right.

The kinds of check

Check onExample ruleCatches
Statusis 200; is 2xx; is 201Failures, wrong success codes
A field's valuestatus equals "paid"Wrong data
A field exists / doesn'tdata.token exists; errors does not existMissing data, hidden errors
A numbertotal is more than 0; quantity is less than 100Nonsense values
A typeprice is a number; items is an array"45" instead of 45
A lengthitems has length > 0Empty lists
A patternid matches ^A-\d+$Badly formed ids, dates, emails
The bodycontains "inStock"; is valid JSONError pages, HTML instead of JSON
A headercontent-type contains json; x-powered-by does not existWrong formats, leaking server details
The timewithin 500 msSlowness
The shapematches a JSON schemaRenamed, missing or retyped fields

How strict should a check be?

Too loose, and broken answers pass. Too strict, and correct answers fail — and people learn to ignore red tests.

  • Check what must be true, not everything that happens to be. items has length > 0 survives a new product being added; items has length 12 breaks the day someone adds one.
  • Don't check values that change every time — timestamps, generated ids, prices that update — for equality. Check their type or pattern instead: createdAt matches ^\d{4}-\d\d-\d\d.
  • Check the thing the test is about. A test of "create an order" should check the order was created (201, an id, the right productId) — not the customer's address.
  • Always check the status, and at least one thing in the body. That pair catches most "fast but wrong" answers, like a 200 with an error inside.
  • Time checks need headroom. If an answer usually takes 80 ms, a 100 ms limit fails on a busy afternoon. Pick a number that means too slow for a person, like 500 ms or 1 s.

Many checks, one answer

A request can have many checks, and each one is reported separately. That matters: "3 of 4 checks passed — $.items[0].price should be number" tells you exactly what broke, where "test failed" doesn't.

When a rule needs code: test scripts

Some rules don't fit a single check: the total is the sum of the line items; every item in the list is in stock; the dates are in order. For those, tools let you write a short test script. In APITestCenter (and Postman) it looks like this:

pm.test("total is the sum of the items", () => {
  const order = pm.response.json();
  const sum = order.items.reduce((s, i) => s + i.price * i.quantity, 0);
  pm.expect(order.total).to.equal(sum);
});

pm.test(name, fn) is one named check; inside it, pm.expect(value).to… states the rule. Reach for scripts when a check can't say it — simple checks are easier to read later.

Checks under load

The same checks can run on every request of a load test. That turns "the API handled 500 requests a second" into "the API handled 500 requests a second and 99.8% of the answers were correct" — because under pressure, APIs don't only slow down; some start answering wrongly: empty lists, timeouts disguised as errors, someone else's data from a broken cache.

Key takeaways

  • A check is a rule a correct answer must follow — status, values, types, lengths, patterns, headers, time, shape.
  • Be strict about what must be true, loose about what may change; check type or pattern for values that vary.
  • Use test scripts for rules a single check can't express — and run your checks under load too.

In the app: Checks · Scripts

Next: Environments and variables

A question these lessons don’t answer? Write to support@apitestcenter.com — we add lessons for the questions people ask.