Automation

From the app to your pipeline

Step by step — build a test suite in the app, put it in your repository, and have every push checked automatically, with a red build when an API breaks or gets slower.

By the end of this page, every push to your repository runs three kinds of checks, and the build goes red when one fails:

CheckQuestion it answersCommand
SuiteDoes every request still work and give correct answers?apitestcenter suite
Performance test with goalsIs it fast enough under load?apitestcenter run --p95 … --errors …
BaselineDid it get slower than before?apitestcenter run --baseline … --max-slower …

Start with the suite — it's the one most teams run on every push. Add the other two when you're ready.

Step 1 — Build the suite in the app

A suite is just a collection of saved requests.

  1. On Test API, build each request and press Ctrl+S to save it into a collection, for example Orders API. (Or import your Postman collection.)
  2. Give each request checks — at least status is 2xx and one JSON field that must be there. A request without checks only fails on 4xx/5xx.
  3. If requests need to sign in, add the sign-in request as Run first. It runs once per suite run.
  4. If one request creates something the next ones need (an order id), use Save values.
  5. Open Run collection, put the requests in order, and press Run until everything is green.

Step 2 — Use variables for anything secret or different per environment

The pipeline can't see secrets saved on your computer, so:

  • Write the server address as {{baseUrl}} and keep its value in an environment such as UAT.
  • Write tokens, passwords, and keys as {{token}}, {{password}} — never typed straight into the request. In the pipeline you'll pass them from your CI's secret store.

Run the collection once more with the UAT environment selected, to be sure every {{variable}} has a value.

Step 3 — Download for CI

On Run collection, with your collection and environment selected, press Download for CI. You get a zip that unzips into the root of your repository:

api-tests/
  requests/Orders API/…json     your saved requests (no secrets)
  environments.json             environment values (secret ones left empty)
  suites.json                   the run order
  README.md                     the command to run, and which secrets it needs
.github/workflows/api-checks.yml   a ready GitHub Actions pipeline

Requests that a request in the collection uses as Run first — even from another collection — are included. If a request still has a secret typed straight into it, the app tells you, and the README lists it: change it to a {{variable}} (step 2) and download again.

Unzip it into your repository and commit it.

Step 4 — Add your secrets to the CI

In GitHub: your repository → Settings → Secrets and variables → Actions → New repository secret, for example ORDERS_TOKEN. Then pass it in .github/workflows/api-checks.yml by adding --var to the command:

run: npx --yes apitestcenter suite "Orders API" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --junit atc-results.xml

--var also overrides any environment value — handy for pointing the same suite at a preview deployment: --var baseUrl=https://pr-123.example.com.

Step 5 — Push, and read the result

Push the commit. In GitHub, open Actions → API checks:

  • Green — every request passed (exit code 0).
  • Red — at least one failed or could not run (exit code 1), or it could not start (exit code 2: a wrong collection or environment name, for example).

The log says exactly what happened, request by request:

suite  Orders API · 4 requests · UAT
  ✓ POST   Sign in        200  182 ms
  ✕ POST   Create order   200   88 ms
      {{orderId}} was not found in the answer (looked for $.id)
  ! GET    Get order
      {{orderId}} should come from "Create order", which failed
  ✓ GET    List products  200   41 ms

FAIL  2 of 4 passed · 1 failed · 1 could not run

The JUnit file atc-results.xml is kept as a build artifact, with one test case per request — test-report tools in GitHub, GitLab, Jenkins, and Azure DevOps show it as a list of passed and failed tests.

Step 6 — Add a performance test with goals (optional)

Pick the request that matters most and add a step that load-tests it, with pass/fail goals:

- name: Speed check
  run: npx --yes apitestcenter run "Orders API/List orders" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --preset smoke --p95 500 --errors 1 --junit atc-speed.xml

It fails when 95% of requests take longer than 500 ms, or more than 1% fail. Start small (--preset smoke, 2 users for 30 seconds) — a pipeline should not hammer a shared environment. k6, the load engine, is downloaded automatically the first time.

Step 7 — Fail when it gets slower (optional)

Goals catch an API that is slow. A baseline catches one that got slower than it was.

  1. Save a good run as the baseline, once, and commit it:

``bash npx apitestcenter run "Orders API/List orders" --data api-tests --env UAT --save-baseline api-tests/baseline.json ``

  1. Compare every later run with it:

```yaml

  • name: Did it get slower? run: npx --yes apitestcenter run "Orders API/List orders" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --preset smoke --baseline api-tests/baseline.json --max-slower 10 --junit atc-baseline.xml ```

It fails when the run is more than 10% slower than the baseline, or fails more often. Use the same load as the baseline — otherwise it's not a fair comparison and is skipped. When a slower speed is expected (a heavier feature), save a new baseline.

The complete pipeline

name: API checks

on:
  push:
  workflow_dispatch:

jobs:
  api-checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Every request works
        run: npx --yes apitestcenter suite "Orders API" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --junit atc-results.xml

      - name: Fast enough, and not slower than before
        run: npx --yes apitestcenter run "Orders API/List orders" --data api-tests --env "UAT" --var token=${{ secrets.ORDERS_TOKEN }} --preset smoke --p95 500 --errors 1 --baseline api-tests/baseline.json --max-slower 10 --junit atc-speed.xml

      - name: Keep the results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: api-check-results
          path: atc-*.xml

GitLab CI and Azure DevOps versions, and every option: Command line and CI.

Keeping it up to date

The app is where you edit; the repository holds a copy. After changing requests, checks, or the run order in the app, press Download for CI again and commit the new files. Review the change in your pull request like any other code.

Something unclear or missing? Write to support@apitestcenter.com — the guide improves with every question.