Automation
From the app to your pipeline
Step by step — build a test suite in the app, put it in your repository, and have every push checked automatically, with a red build when an API breaks or gets slower.
By the end of this page, every push to your repository runs three kinds of checks, and the build goes red when one fails:
| Check | Question it answers | Command |
|---|---|---|
| Suite | Does every request still work and give correct answers? | apitestcenter suite |
| Performance test with goals | Is it fast enough under load? | apitestcenter run --p95 … --errors … |
| Baseline | Did it get slower than before? | apitestcenter run --baseline … --max-slower … |
Start with the suite — it's the one most teams run on every push. Add the other two when you're ready.
Step 1 — Build the suite in the app
A suite is just a collection of saved requests.
- On Test API, build each request and press Ctrl+S to save it into a collection, for example Orders API. (Or import your Postman collection.)
- Give each request checks — at least status is 2xx and one JSON field that must be there. A request without checks only fails on 4xx/5xx.
- If requests need to sign in, add the sign-in request as Run first. It runs once per suite run.
- If one request creates something the next ones need (an order id), use Save values.
- Open Run collection, put the requests in order, and press Run until everything is green.
Step 2 — Use variables for anything secret or different per environment
The pipeline can't see secrets saved on your computer, so:
- Write the server address as
{{baseUrl}}and keep its value in an environment such as UAT. - Write tokens, passwords, and keys as
{{token}},{{password}}— never typed straight into the request. In the pipeline you'll pass them from your CI's secret store.
Run the collection once more with the UAT environment selected, to be sure every {{variable}} has a value.
Step 3 — Download for CI
On Run collection, with your collection and environment selected, press Download for CI. You get a zip that unzips into the root of your repository:
api-tests/
requests/Orders API/…json your saved requests (no secrets)
environments.json environment values (secret ones left empty)
suites.json the run order
README.md the command to run, and which secrets it needs
.github/workflows/api-checks.yml a ready GitHub Actions pipeline
Requests that a request in the collection uses as Run first — even from another collection — are included. If a request still has a secret typed straight into it, the app tells you, and the README lists it: change it to a {{variable}} (step 2) and download again.
Unzip it into your repository and commit it.
Step 4 — Add your secrets to the CI
In GitHub: your repository → Settings → Secrets and variables → Actions → New repository secret, for example ORDERS_TOKEN. Then pass it in .github/workflows/api-checks.yml by adding --var to the command:
run: npx --yes apitestcenter suite "Orders API" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --junit atc-results.xml
--var also overrides any environment value — handy for pointing the same suite at a preview deployment: --var baseUrl=https://pr-123.example.com.
Step 5 — Push, and read the result
Push the commit. In GitHub, open Actions → API checks:
- Green — every request passed (exit code 0).
- Red — at least one failed or could not run (exit code 1), or it could not start (exit code 2: a wrong collection or environment name, for example).
The log says exactly what happened, request by request:
suite Orders API · 4 requests · UAT
✓ POST Sign in 200 182 ms
✕ POST Create order 200 88 ms
{{orderId}} was not found in the answer (looked for $.id)
! GET Get order
{{orderId}} should come from "Create order", which failed
✓ GET List products 200 41 ms
FAIL 2 of 4 passed · 1 failed · 1 could not run
The JUnit file atc-results.xml is kept as a build artifact, with one test case per request — test-report tools in GitHub, GitLab, Jenkins, and Azure DevOps show it as a list of passed and failed tests.
Step 6 — Add a performance test with goals (optional)
Pick the request that matters most and add a step that load-tests it, with pass/fail goals:
- name: Speed check
run: npx --yes apitestcenter run "Orders API/List orders" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --preset smoke --p95 500 --errors 1 --junit atc-speed.xml
It fails when 95% of requests take longer than 500 ms, or more than 1% fail. Start small (--preset smoke, 2 users for 30 seconds) — a pipeline should not hammer a shared environment. k6, the load engine, is downloaded automatically the first time.
Step 7 — Fail when it gets slower (optional)
Goals catch an API that is slow. A baseline catches one that got slower than it was.
- Save a good run as the baseline, once, and commit it:
``bash npx apitestcenter run "Orders API/List orders" --data api-tests --env UAT --save-baseline api-tests/baseline.json ``
- Compare every later run with it:
```yaml
- name: Did it get slower? run: npx --yes apitestcenter run "Orders API/List orders" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --preset smoke --baseline api-tests/baseline.json --max-slower 10 --junit atc-baseline.xml ```
It fails when the run is more than 10% slower than the baseline, or fails more often. Use the same load as the baseline — otherwise it's not a fair comparison and is skipped. When a slower speed is expected (a heavier feature), save a new baseline.
The complete pipeline
name: API checks
on:
push:
workflow_dispatch:
jobs:
api-checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Every request works
run: npx --yes apitestcenter suite "Orders API" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --junit atc-results.xml
- name: Fast enough, and not slower than before
run: npx --yes apitestcenter run "Orders API/List orders" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --preset smoke --p95 500 --errors 1 --baseline api-tests/baseline.json --max-slower 10 --junit atc-speed.xml
- name: Keep the results
if: always()
uses: actions/upload-artifact@v4
with:
name: api-check-results
path: atc-*.xml
GitLab CI and Azure DevOps versions, and every option: Command line and CI.
Keeping it up to date
The app is where you edit; the repository holds a copy. After changing requests, checks, or the run order in the app, press Download for CI again and commit the new files. Review the change in your pull request like any other code.