Testing APIs
Checks
Rules for a correct answer. A fast answer with an error inside should still fail — checks make sure it does.
Open the Checks tab and add rules. Each Send shows ✓ or ✕ for every rule, and a performance test applies them to every request, so the report says what share of answers were correct — not just fast.
Kinds of checks
| Check | Passes when | Example |
|---|---|---|
| Status | The status matches. Use one code, several, a range, or a class. | 200 · 200, 201 · 200-299 · 2xx |
| JSON field | A field exists or not; equals, does not equal, or contains a value; is more or less than a number; matches a pattern; is of a type; has a length. | data.items[0].id exists · errors does not exist · total is more than 0 · items has length >0 · id is of type integer |
| Body text | The answer contains — or does not contain — some text, matches a pattern, or is valid JSON. | does not contain error · is valid JSON |
| Header | A response header exists or not, equals, contains, or matches a pattern. | content-type contains json · x-powered-by does not exist |
| Response time | The answer arrived within a time, in milliseconds. | within 500 ms |
| JSON schema | The whole answer matches a JSON Schema. | see below |
JSON paths start from the top of the answer: name, data.user.id, items[2].price. A leading $. is fine too.
Types are string, number, integer (a whole number), boolean, array, object and null. Length counts the items of a list, the characters of a text, or the keys of an object; write 3, >0, >=1, <10, <=10 or !=0. Patterns are regular expressions, like ^ord_\d+$.
When a check fails, Send says why next to it — got 404, is "pending", not found — and so does the collection run, in the app and in CI.
Compare with a value from another request
A check's expected value can be a {{name}}: a value an earlier request saved, a test-data column or an environment variable. That's how you check that data arrived correctly in another system. For example, a collection run that:
- Reads an employee from SuccessFactors and saves
$.d.results[0].userIdasemployeeId(Save values). - Reads the same person from NetSuite, with a check: JSON
externalIdequals{{employeeId}}.
Step 2 passes only when NetSuite has the value SuccessFactors gave. When it differs, the check shows what NetSuite had. When nothing saved employeeId (the first request failed, or this request was sent on its own), the check fails with {{employeeId}} has no value rather than comparing with the text {{employeeId}}.
{{name}} works in every check's value, and in a JSON path or header name too.
Field mapping
When a record goes from one system to another, many fields have to arrive correctly at once, and each system writes them its own way. Checks → Add Field mapping checks them all in one go:
- In the request that reads the source system, Save values → JSON field
$(ord.results[0]for SAP OData) as{{employee}}. - In the request that reads the target system, add Field mapping. Add the last answer's fields lists the record's fields, one row each, after you Send once.
- For each row, type what it should be:
{{employee.firstName}}, and pick how to compare:
- Exact: the same text.
- Ignore case:
De Souzamatchesde souza. - Number:
5200matches"5200.00". - Same day: any date format:
2024-01-31,31.01.2024,1/31/2024(month first with slashes) and SAP's/Date(1706659200000)/all match each other.
- Translate changes the expected value before comparing, when the systems use different codes:
M=Male, F=Female.
It passes when every row matches. When some don't, it lists them: 2 of 14 differ: $.firstname is "Anna", should be "Ana"; $.hiredate not found. In a collection run, a step whose source request failed says which value has no value instead.
XML and SOAP answers
JSON field checks, Save values and Field mapping read XML answers too, with the same paths. Element names lose their prefix (soap:Envelope is Envelope), so this SOAP answer:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GetEmployeeResponse>
<Employee id="E1004">
<FirstName>Ben</FirstName>
<Salary currency="EUR">52000.00</Salary>
<Phone>+1 555 0100</Phone>
<Phone>+1 555 0101</Phone>
</Employee>
</GetEmployeeResponse>
</soap:Body>
</soap:Envelope>
reads as:
Envelope.Body.GetEmployeeResponse.Employee.FirstNameisBen.Envelope.Body.GetEmployeeResponse.Employee.@idisE1004: an attribute is@and its name.Envelope.Body.GetEmployeeResponse.Employee.Salary.#textis52000.00: the text of an element that also has attributes is#text.Envelope.Body.GetEmployeeResponse.Employee.Phone[1]is the second phone. An element that appears more than once is a list;[0]also works when there's only one, so a path works whether the answer has one item or several.- An element with
xsi:nil="true"is empty (null).
A JSON schema check still needs a JSON answer.
To try it, the practice API has a SOAP service: send POST {{practiceApi}}/soap/employees with Basic sign-in (any user name, the password demo), or import its WSDL from {{practiceApi}}/soap/employees?wsdl (see Importing requests). Its GetEmployee gives the same people as the practice SuccessFactors service.
JSON schema
A schema check makes sure every answer has the right shape: the fields that must be there, and the type of each — a whole contract in one check, instead of one check per field.
- Send the request once.
- Checks → Add JSON schema → Make it from the last response. It writes a schema the answer matches: every field it had is required, with its type. A list's items take the fields every item had.
- Loosen or tighten it — remove a field from
requiredwhen it may be missing, add"nullable": trueor"type": ["string", "null"]when it may be empty, add"enum","minimum"or"pattern"for allowed values.
When an answer differs, the check says where: $.items[0].id should be integer (it is string), $ should have id.
Supported: type (with nullable), enum, const, properties, required, additionalProperties, items, minItems/maxItems, minLength/maxLength, pattern, minimum/maximum and their exclusive forms, allOf/anyOf/oneOf, and $ref to the schema's own definitions or $defs. Other keywords — format, say — are ignored. Importing an OpenAPI or Swagger file adds a schema check like this to each request, from the file's response schemas ($refs to its own components work; outside files don't) — or, by default, keeps the file as the collection's API description, which checks every answer against it, error answers too: see Contract tests.
In a performance test the schema is checked on every answer; a large schema on a fast API costs a little CPU on this computer.
How checks affect results
- Send: the Checks badge shows how many passed (for example 2/3), and each rule is listed in the response.
- Performance test: the report shows the share of passing checks and which check failed most often. By default the result fails if fewer than 99% of checks pass; you can change that under Goals.
- Collection run: a request with checks passes only when all of them pass. A request without checks passes on any 2xx or 3xx status.