feat: add API automation tests and contract check scripts

This commit is contained in:
2026-06-10 20:44:10 -07:00
parent 1fc05b7b3a
commit b692affdc8
5 changed files with 247 additions and 0 deletions

View File

@@ -0,0 +1,60 @@
# API Automation Tests
This project has two API automation layers:
- `scripts/api-contract-test.sh` runs OpenAPI contract checks with Schemathesis against the schema at `OPENAPI_SCHEMA_URL`.
- `tests/auth-live-api.test.ts` runs focused Vitest live scenarios against a deployed or local Better Auth API.
## Local Commands
Run repository tests without network dependencies:
```bash
pnpm test
```
Run live smoke scenarios:
```bash
API_BASE_URL=https://cfw-auth.bowong.cc/api/auth pnpm test tests/auth-live-api.test.ts
```
Run OpenAPI contract checks:
```bash
API_BASE_URL=https://cfw-auth.bowong.cc/api/auth pnpm api:test:contract
```
Use a lower `MAX_EXAMPLES` for pull requests and a higher value for scheduled checks:
```bash
API_BASE_URL=https://cfw-auth.bowong.cc/api/auth MAX_EXAMPLES=100 pnpm api:test:contract
```
## Environment
- `API_BASE_URL`: required for live and contract tests. Use the auth base path, for example `https://cfw-auth.bowong.cc/api/auth`.
- `OPENAPI_SCHEMA_URL`: optional schema URL. Defaults to `https://cfw-auth.bowong.cc/api/auth/open-api/generate-schema`.
- `OPENAPI_SCHEMA`: optional downloaded schema path. Defaults to `reports/openapi/cfw-auth-openapi.json`.
- `API_TOKEN`: optional bearer token for protected checks.
- `RUN_ID`: optional namespace for test data. Defaults to a timestamp-based value.
- `ALLOW_AUTH_WRITE_TESTS=true`: enables Vitest scenarios that create authentication data.
- `MAX_EXAMPLES`, `TEST_SEED`, `REQUEST_TIMEOUT`, `SCHEMATHESIS_PHASES`, `SCHEMATHESIS_CHECKS`, `REPORT_DIR`: tune Schemathesis execution.
## Coverage
OpenAPI automation covers static schema linting and schema-driven request/response checks. The default Schemathesis phases are `examples,coverage` to avoid broad fuzzing against production by accident.
Vitest live tests currently cover:
- OpenAPI document availability and expected core paths.
- `GET /ok` readiness response.
- anonymous `GET /get-session` behavior.
- invalid email sign-in rejection.
- optional namespaced email sign-up when write tests are explicitly enabled.
## Safety
The OpenAPI document includes account, organization, session, API key, and admin write operations. Do not run broad fuzzing or stateful phases against production unless the target is an isolated test environment.
Write scenarios are skipped unless `ALLOW_AUTH_WRITE_TESTS=true`. Generated users use `RUN_ID` and the `example.invalid` domain, but there is no public delete-user cleanup in the default scenario because Better Auth deletion normally requires a valid session and may require verification.