API Test Files

Write API tests as YAML files in your repository, run them with openpost test, and reuse the same files for load tests, documentation and CI.

CLI

Why test files

An API test file (*.api.yaml) is a plain YAML file that lives in your repository next to the code it tests. It replaces the one-off Python, JavaScript or curl scripts people write to poke at a local API: the same file is the test, the load test, the documentation source and the CI check.

Reviewable — diffs read like the API change they cover

No secrets in git — tokens come from {{$env.NAME}} or a gitignored .env

Editor autocomplete — a JSON Schema ships with the CLI (openpost schema)

Works for AI agents — see AI Agent Skills

Quick Start

npx openpost init

Creates api-tests/example.api.yaml, .env.example, a CI workflow and .gitignore entries

npx openpost send GET http://localhost:3000/health -e status=200

One request with an assertion — a scriptable curl with checks

npx openpost send GET localhost:3000/tasks -e status=200 -e "$.length>0" \
  --save api-tests/tasks.api.yaml --name "List tasks"

Saves the request into a test file; literal tokens are moved to {{$env.NAME}}

npx openpost test

Runs every file under ./api-tests

File Format

A file describes one API or resource. Requests run in order, so later requests can use values captured by earlier ones.

# api-tests/tasks.api.yaml
name: Tasks API
description: CRUD over tasks, protected by a bearer token.
baseUrl: "{{baseUrl}}"
vars:
  baseUrl: http://localhost:3000
environments:
  local:   { baseUrl: http://localhost:3000 }
  staging: { baseUrl: https://staging.example.com }
defaults:
  auth: { bearer: "{{token}}" }
requests:
  - name: Login
    method: POST
    url: /auth/login
    auth: null
    once: true                      # load tests: run once, share the token
    json: { username: admin, password: "{{$env.API_PASSWORD}}" }
    expect: { status: 200, body: { $.token: { type: string } } }
    capture: { token: $.token }

  - name: Create task
    method: POST
    url: /tasks
    json: { title: "Write docs {{$randomInt(1,999)}}" }
    expect:
      status: 201
      time: "< 500"
      body:
        $.id: { type: integer }
        $.done: false
    capture: { taskId: $.id }

  - name: Get task
    url: /tasks/{{taskId}}
    expect:
      status: 200
      schema:
        type: object
        required: [id, title, done]

  - name: Unauthorized without token
    url: /tasks
    auth: null
    expect: { status: 401 }

Request key	Meaning
name	Unique within the file; used by --grep, reports and docs
method / url	GET by default; url is absolute or relative to baseUrl
query / headers	Maps; headers merge over defaults.headers
json / body / form / graphql	One request body type
auth	bearer, basic or apiKey; null turns off defaults.auth
expect	Assertions (see below). Without expect.status any status ≥ 400 fails
capture	name: selector — saved for later requests
data	Inline rows or a .csv/.json file: run the request once per row
retries / delay / timeout	Per-request overrides, e.g. polling for eventual consistency
once	Load tests only: run once before the load (login, setup)
tags / skip / only	Filtering
doc	Markdown shown in generated docs

Variables & Secrets

{{name}} resolves from vars, then the selected environment, then --env-file, then --var name=value, then the data row, then captures. {{$env.NAME}} reads a process environment variable; values in --env-file fill both forms.

Built-in	Value
{{$uuid}}	Random UUID
{{$timestamp}} / {{$isoTimestamp}}	Current time
{{$randomInt(1,100)}}	Random integer in range
{{$randomEmail}} / {{$randomString(12)}}	Random test data
{{$iteration}}	Data row or load-test iteration index

A request with an unresolved variable is not sent; the error names the variable and where to define it

If a captured value is missing because an earlier request failed, the error names that request

Reports mask secret values; --show-secrets turns masking off for local debugging

Assertions

Every failed assertion prints the selector, the rule and the actual value. Selectors are JSONPath-style: $.user.name, $.items[0], $.items[*].id, $..id, $.items.length, header.location.

expect:
  status: 201                  # or [200, 201], "2xx", "< 400"
  time: "< 500"                # total ms
  headers:
    content-type: { contains: json }
  body:
    $.id: { type: integer, gt: 0 }
    $.status: active           # literal = equality
    $.email: "/^[^@]+@example\\.com$/"
    $.items: { type: array, length: { between: [1, 50] } }
    $.items[*].price: { every: { gt: 0 } }
    $.deletedAt: { exists: false }
  schema:                      # JSON Schema for the whole body
    type: object
    required: [id, items]

Matcher	Passes when
eq, ne	Equal / not equal
gt, gte, lt, lte, between	Numeric comparison
contains, notContains	Substring, array element or partial object
startsWith, endsWith, matches	String prefix, suffix or regex
in, notIn	Value is (not) in a list
exists, empty, type, length	Presence, emptiness, JSON type, size
schema	JSON Schema for this value
every, some	Matcher for every / at least one array item

Running Tests

openpost validate api-tests/                      # schema + typo check, no requests sent
openpost test api-tests/ --env staging           # run everything against staging
openpost test --grep "Get task"                   # one request plus the ones it depends on
openpost test --env-file api-tests/.env -r html,junit --report-dir reports
openpost test --json > result.json                # machine-readable result

Exit code	Meaning
0	All assertions passed
1	Assertions or thresholds failed
2	Invalid file, bad flags, or nothing matched

--grep matches names by substring, glob (Get*) or /regex/, and automatically includes earlier requests whose captures the selected ones need. validate treats unknown keys and typos as errors and suggests the key you probably meant.

Documentation from Tests

openpost docs api-tests/ --format md      --out docs/API.md
openpost docs api-tests/ --format html    --out docs/api.html --run   # embeds real example responses
openpost docs api-tests/ --format openapi --out openapi.json

Add a description to each file and a Markdown doc field to requests. One file per resource gives one docs section each, and request names like "Create task" read as operations. OpenAPI output merges requests with the same path shape (for example /tasks/1 and /tasks/{{id}}) and lists negative cases as extra responses.

Working with Collections

openpost export tests "My Collection" -o api-tests/my-collection.api.yaml
openpost import tests api-tests/tasks.api.yaml

Export turns a collection built in the app into a test file, moving secrets to {{$env.*}}; import brings a test file back into the app. GUI test rules and set-variable rules map to expect and capture.

Reports & Metrics → Load Testing → AI Agent Skills →