93 lines
3.9 KiB
Markdown
93 lines
3.9 KiB
Markdown
# API Automation Tests
|
|
|
|
This project has two API automation layers:
|
|
|
|
- `scripts/api-contract-test.sh` runs OpenAPI contract checks with Schemathesis against the schema at `OPENAPI_SCHEMA_URL`.
|
|
- `tests/auth-live-api.test.ts` runs focused Vitest live scenarios against a deployed or local Better Auth API.
|
|
- `tests/auth-business-scenarios.test.ts` runs fixture-backed business workflows that create, update, and delete test data.
|
|
|
|
## Local Commands
|
|
|
|
Run repository tests without network dependencies:
|
|
|
|
```bash
|
|
pnpm test
|
|
```
|
|
|
|
Create or refresh writable live-test fixtures:
|
|
|
|
```bash
|
|
pnpm api:test:bootstrap
|
|
```
|
|
|
|
Run live smoke scenarios:
|
|
|
|
```bash
|
|
pnpm api:test:live
|
|
```
|
|
|
|
Run business workflow scenarios:
|
|
|
|
```bash
|
|
pnpm api:test:business
|
|
```
|
|
|
|
Run OpenAPI contract checks:
|
|
|
|
```bash
|
|
API_BASE_URL=https://cfw-auth.bowong.cc/api/auth pnpm api:test:contract
|
|
```
|
|
|
|
Use a lower `MAX_EXAMPLES` for pull requests and a higher value for scheduled checks:
|
|
|
|
```bash
|
|
API_BASE_URL=https://cfw-auth.bowong.cc/api/auth MAX_EXAMPLES=100 pnpm api:test:contract
|
|
```
|
|
|
|
## Environment
|
|
|
|
- `API_BASE_URL`: required for live and contract tests. Use the auth base path, for example `https://cfw-auth.bowong.cc/api/auth`.
|
|
- `OPENAPI_SCHEMA_URL`: optional schema URL. Defaults to `https://cfw-auth.bowong.cc/api/auth/open-api/generate-schema`.
|
|
- `OPENAPI_SCHEMA`: optional downloaded schema path. Defaults to `reports/openapi/cfw-auth-openapi.json`.
|
|
- `API_TOKEN`: optional bearer token for protected checks.
|
|
- `RUN_ID`: optional namespace for test data. Defaults to a timestamp-based value.
|
|
- `ALLOW_AUTH_WRITE_TESTS=true`: enables Vitest scenarios that create authentication data.
|
|
- `MAX_EXAMPLES`, `TEST_SEED`, `REQUEST_TIMEOUT`, `SCHEMATHESIS_PHASES`, `SCHEMATHESIS_CHECKS`, `REPORT_DIR`: tune Schemathesis execution.
|
|
|
|
`pnpm api:test:bootstrap` writes `.api-test.env`, which is ignored by git. It contains generated passwords, session cookies, API keys, and fixture IDs for later live tests:
|
|
|
|
- `API_TEST_USER_ID`, `API_TEST_USER_EMAIL`, `API_TEST_SESSION_COOKIE`
|
|
- `API_TEST_ADMIN_ID`, `API_TEST_ADMIN_EMAIL`, `API_TEST_ADMIN_SESSION_COOKIE`
|
|
- `API_TEST_ORGANIZATION_ID`, `API_TEST_ORGANIZATION_SLUG`
|
|
- `API_TEST_API_KEY_ID`, `API_TEST_API_KEY`
|
|
|
|
The bootstrap script creates users through the public auth API, then uses Wrangler D1 access to mark only those namespaced test users as email-verified and to assign the test admin role. This keeps administrator bootstrap outside the public auth API.
|
|
|
|
## Coverage
|
|
|
|
OpenAPI automation covers static schema linting and schema-driven request/response checks. The default Schemathesis phases are `examples,coverage` to avoid broad fuzzing against production by accident.
|
|
|
|
Vitest live tests currently cover:
|
|
|
|
- OpenAPI document availability and expected core paths.
|
|
- `GET /ok` readiness response.
|
|
- anonymous `GET /get-session` behavior.
|
|
- invalid email sign-in rejection.
|
|
- optional namespaced email sign-up when write tests are explicitly enabled.
|
|
- bootstrapped user session, admin user lookup, organization listing, and API key listing when `.api-test.env` exists.
|
|
|
|
Business scenario tests currently cover:
|
|
|
|
- Email sign-up requiring verification before password sign-in.
|
|
- Verified user sign-in, session read, and sign-out.
|
|
- Admin create/read/update/ban/unban/remove user lifecycle.
|
|
- Regular user rejection from admin endpoints.
|
|
- Organization active selection plus team create/list-members/update/remove workflow.
|
|
- API key create/read/server-only update rejection/delete workflow.
|
|
|
|
## Safety
|
|
|
|
The OpenAPI document includes account, organization, session, API key, and admin write operations. Do not run broad fuzzing or stateful phases against production unless the target is an isolated test environment.
|
|
|
|
Write scenarios are intended for the current test environment. Generated users use `RUN_ID` and the `example.invalid` domain. Business scenarios create temporary users, teams, and API keys, then remove the resources whose lifecycle is part of the scenario.
|