Testing APIs
Contracts and schemas
An API's contract is its promise about requests and answers. OpenAPI describes it; JSON Schema checks answers against it. How contract testing catches breaking changes before the apps built on the API do.
When a mobile app shows an order, it expects total to be a number and items to be a list. If the API one day sends "total": "45.50" or renames items to lines, the app crashes — even though the API still answers 200 and the data is "there". The API broke its contract.
An API's contract is its promise: which requests it accepts, and what shape each answer has — the fields, their types, which are always present, which values are allowed. Contract testing checks the API still keeps it.
OpenAPI: the contract, written down
Most REST APIs describe their contract in an OpenAPI file (older ones say Swagger) — YAML or JSON listing every path, method, parameter and answer:
paths:
/orders/{id}:
get:
summary: Get an order
responses:
"200":
content:
application/json:
schema:
$ref: "#/components/schemas/Order"
components:
schemas:
Order:
type: object
required: [id, status, total, items]
properties:
id: { type: string, pattern: "^A-\\d+$" }
status: { type: string, enum: [new, paid, shipped, cancelled] }
total: { type: number, minimum: 0 }
items: { type: array, items: { $ref: "#/components/schemas/Item" } }
The file is often the API's documentation page too (it's what tools like Swagger UI render). Importing it into a testing tool gives you a request for every operation at once.
JSON Schema: checking an answer's shape
The schema: part above is a JSON Schema — a JSON (or YAML) description of what valid data looks like. It can say:
| Keyword | Means | Example |
|---|---|---|
type | the kind of value | "type": "number" |
required | fields that must be there | ["id", "status"] |
properties | the fields and their own schemas | { "total": { "type": "number" } } |
items | the schema of each entry in a list | { "$ref": "#/…/Item" } |
enum | the only allowed values | ["new", "paid", "shipped"] |
minimum, maximum | number limits | "minimum": 0 |
minLength, pattern | text limits | "pattern": "^A-\\d+$" |
nullable / ["string", "null"] | may be empty | |
additionalProperties: false | no fields beyond these |
A schema check runs every answer through the schema and reports the first place it differs: $.items[0].price should be number (it is string), $ should have total. One schema check replaces dozens of single-field checks — and keeps checking fields nobody thought to write a check for.
What breaks a contract
Breaking changes (the apps built on the API must change too):
- removing or renaming a field; changing its type (
45→"45"); making it sometimes missing ornull; - adding a new required request field;
- removing an allowed value from an
enum, or narrowing a limit; - changing a status code (
201→200), or an error's format.
Usually safe changes: adding a new optional field to an answer, a new endpoint, a new optional parameter. (That's why a schema shouldn't forbid extra answer fields unless you mean it.)
Two ways to get a schema
- From the API's OpenAPI file — the documented contract. Testing against it finds where the implementation differs from the documentation — and sometimes the documentation is what's wrong.
- From a real answer — tools can write a schema from a response you have: every field it had, with its type. Then you loosen it (this field may be missing; this one may be null) or tighten it (this must be one of these values). Good for APIs with no documentation.
Checking a whole API against its description
A schema check covers one request. With the API's OpenAPI file, a tool can check every answer against it, with no checks written by hand:
- Is the operation in the description? A request to a path or method it doesn't list is either a typo, or an endpoint nobody documented.
- Is the status documented? An undocumented
500— or a200where it promised201— fails. - Is the content type, and does the body match the documented schema? For error answers too.
The same file also shows coverage: which operations have a test, and which have none yet. An API with 97 operations and tests for 40 has 57 nobody checks — the description lists them for you.
Consumer-driven contracts, briefly
In bigger organisations, the teams that use an API (the consumers) write down what they rely on, and the API team's pipeline checks every change against all consumers' expectations (tools like Pact do this). It's the same idea at a larger scale: catch the breaking change in the API's pipeline, not in production.
Key takeaways
- An API's contract is the shape it promises; OpenAPI writes it down, JSON Schema checks answers against it.
- A schema check reports exactly where an answer differs, and covers every field at once.
- Breaking changes — removed, renamed or retyped fields — are what contract tests catch before the apps built on the API break.
- Checking every answer against the API's description needs no hand-written checks, and shows which operations have no test yet.
In the app: Contract tests against an OpenAPI file · JSON schema checks · Importing OpenAPI