cfw-auth/docs/api-automation-tests.md

API Automation Tests

This project has two API automation layers:

• scripts/api-contract-test.sh runs OpenAPI contract checks with Schemathesis against the schema at OPENAPI_SCHEMA_URL.
• tests/auth-live-api.test.ts runs focused Vitest live scenarios against a deployed or local Better Auth API.

Local Commands

Run repository tests without network dependencies:

pnpm test

Create or refresh writable live-test fixtures:

pnpm api:test:bootstrap

Run live smoke scenarios:

pnpm api:test:live

Run OpenAPI contract checks:

API_BASE_URL=https://cfw-auth.bowong.cc/api/auth pnpm api:test:contract

Use a lower MAX_EXAMPLES for pull requests and a higher value for scheduled checks:

API_BASE_URL=https://cfw-auth.bowong.cc/api/auth MAX_EXAMPLES=100 pnpm api:test:contract

Environment

• API_BASE_URL: required for live and contract tests. Use the auth base path, for example https://cfw-auth.bowong.cc/api/auth.
• OPENAPI_SCHEMA_URL: optional schema URL. Defaults to https://cfw-auth.bowong.cc/api/auth/open-api/generate-schema.
• OPENAPI_SCHEMA: optional downloaded schema path. Defaults to reports/openapi/cfw-auth-openapi.json.
• API_TOKEN: optional bearer token for protected checks.
• RUN_ID: optional namespace for test data. Defaults to a timestamp-based value.
• ALLOW_AUTH_WRITE_TESTS=true: enables Vitest scenarios that create authentication data.
• MAX_EXAMPLES, TEST_SEED, REQUEST_TIMEOUT, SCHEMATHESIS_PHASES, SCHEMATHESIS_CHECKS, REPORT_DIR: tune Schemathesis execution.

pnpm api:test:bootstrap writes .api-test.env, which is ignored by git. It contains generated passwords, session cookies, API keys, and fixture IDs for later live tests:

• API_TEST_USER_ID, API_TEST_USER_EMAIL, API_TEST_SESSION_COOKIE
• API_TEST_ADMIN_ID, API_TEST_ADMIN_EMAIL, API_TEST_ADMIN_SESSION_COOKIE
• API_TEST_ORGANIZATION_ID, API_TEST_ORGANIZATION_SLUG
• API_TEST_API_KEY_ID, API_TEST_API_KEY

The bootstrap script creates users through the public auth API, then uses Wrangler D1 access to mark only those namespaced test users as email-verified and to assign the test admin role. This keeps administrator bootstrap outside the public auth API.

Coverage

OpenAPI automation covers static schema linting and schema-driven request/response checks. The default Schemathesis phases are examples,coverage to avoid broad fuzzing against production by accident.

Vitest live tests currently cover:

• OpenAPI document availability and expected core paths.
• GET /ok readiness response.
• anonymous GET /get-session behavior.
• invalid email sign-in rejection.
• optional namespaced email sign-up when write tests are explicitly enabled.
• bootstrapped user session, admin user lookup, organization listing, and API key listing when .api-test.env exists.

Safety

The OpenAPI document includes account, organization, session, API key, and admin write operations. Do not run broad fuzzing or stateful phases against production unless the target is an isolated test environment.

Write scenarios are skipped unless ALLOW_AUTH_WRITE_TESTS=true. Generated users use RUN_ID and the example.invalid domain, but there is no public delete-user cleanup in the default scenario because Better Auth deletion normally requires a valid session and may require verification.