comi-logic/cfw-logging
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: import logging service

Browse Source
This commit is contained in:
杨明明
2026-07-14 00:12:51 -07:00
commit a9a2abbb47
54 changed files with 18955 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

249
README.md Normal file
View File

@@ -0,0 +1,249 @@
# Logger System
Independent TypeScript logger collection system with:
- `@logger/sdk`: application logger API and transports.
- `@logger/collector`: JSONL store, HTTP ingest, and read-only HTTP query.
- `@logger/cli`: `log tail` / `log config` for reading records, driven by a `.logging/config.json` profile.
- `logging-agent` skill: a usage contract for Codex / Claude Code to read logs safely.
## Install
```bash
pnpm install
pnpm test
pnpm build
```
## SDK
```ts
import {createLogger, HttpTransport} from '@logger/sdk';
const log = createLogger({
app: 'shop',
service: 'orders',
scope: 'orders.refund',
runtime: 'server',
transport: new HttpTransport('http://127.0.0.1:4319/ingest'),
});
log.info('hello');
log.error('error msg', new Error('payment timeout'), {orderId: 'o-2099'});
await log.flush();
```
Records are structured JSON with `timestamp`, `level`, `message`, `app`, `service`, `scope`, `runtime`, optional `attributes`, optional serialized `error`, and `version: 1`.
Sensitive fields are redacted by default when passed in attributes:
- `password`
- `token`
- `secret`
- `apiKey`
- `authorization`
- `cookie`
- `set-cookie`
## Collector
Use `JsonlLogStore` directly:
```ts
import {JsonlLogStore} from '@logger/collector';
const store = new JsonlLogStore('.logging/logs.jsonl');
await store.append(record);
const latest = await store.tail({limit: 10, level: 'error'});
```
Start an HTTP collector server from application code:
```ts
import {createHttpCollectorServer, JsonlLogStore} from '@logger/collector';
const store = new JsonlLogStore('.logging/logs.jsonl');
createHttpCollectorServer(store).listen(4319);
```
Endpoints:
- `POST /ingest` — append a single `LogRecord` or an array of `LogRecord` objects.
- `GET /query` — read-only tail. Supports query params `level`, `app`, `service`, `scope`, `since`, `limit`; returns a JSON array of matching records (last `limit`, file order).
## Configuration (`.logging/config.json`)
The CLI is driven by a project-local profile at `.logging/config.json`. It declares the system name, where logs live, optional remote access, and safe defaults. Secrets are never stored in the file — only the name of the environment variable that holds a token.
```json
{
"version": 1,
"systemName": "shop",
"local": {"file": ".logging/logs.jsonl"},
"server": {"baseUrl": "https://logs.example.com", "queryPath": "/query", "ingestPath": "/ingest"},
"auth": {"type": "bearer", "tokenEnv": "LOGGING_TOKEN"},
"defaults": {"app": "shop", "service": "orders", "limit": 50},
"output": {"format": "compact", "redact": true, "maxLines": 200}
}
```
Required: `version` (must be `1`) and `systemName`. Everything else is optional. See `skills/logging-agent/references/config-contract.md` for the full field reference and security rules.
### Bootstrapping a profile
```bash
log config init --system=shop
log config init --system=shop --server=https://logs.example.com --token-env=LOGGING_TOKEN
```
`config init` writes `.logging/config.json` plus a `.logging/.gitignore` that ignores `*.local.json`, `*.secret.json`, `tokens*`, and `logs.jsonl` while keeping `config.json` committable. Pass `--force` to overwrite an existing profile.
### Checking a profile
```bash
log config doctor
```
Validates the config shape, confirms `systemName`, checks that any required token environment variable is set (without printing its value), probes remote reachability when a server is configured, and checks the local file. Exits non-zero if any hard check fails.
## CLI
After build:
```bash
node packages/cli/dist/index.js tail
node packages/cli/dist/index.js tail --level=error --app=shop --json
node packages/cli/dist/index.js tail --remote --json
```
The package binary name is `log`:
```bash
log tail
log tail --level=error --app=shop --service=orders --scope=orders.refund --json
log config init --system=shop
log config doctor
```
### Where `tail` reads from
`tail` resolves its source predictably and never silently calls a remote service:
1. `--remote` → query the configured `server` (requires `server.baseUrl`; auth resolved from `auth`).
2. `--file=<path>` or `LOGGER_FILE` → read that local JSONL file.
3. `config.local.file` → read the configured local file.
4. `config.server` configured with no local source → query the server.
5. fallback → read `.logging/logs.jsonl` and print a hint to run `log config init`.
### Filter and output resolution
Command-line flags override `config.defaults`, which override the built-in defaults. Supported filters: `--level`, `--app`, `--service`, `--scope`, `--since`, `-n`/`--limit`. Output is capped to `config.output.maxLines` (default `200`); `--json` emits one record per line; `output.redact` (default `true`) masks known sensitive fields as a safety net even for records that were not redacted at ingest.
Missing config, missing token, or a missing query endpoint each produce a clear error message instead of a silent guess.
## Publishing (private Verdaccio)
Build first, then publish with the helper script:
```bash
pnpm build
./scripts/publish.sh # version defaults to packages/shared-schema's version
./scripts/publish.sh 0.2.2 # or pass an explicit version
```
Use the script instead of `pnpm publish -r`. Verdaccio v6 does **not** extract README from the tarball — it only reads the manifest `readme` field, which modern `npm`/`pnpm publish` no longer populate, so a plain publish makes the registry UI show "No README data found!". The script injects `readme` into each published manifest (rewriting `workspace:*` to the real version on the way, then restoring `package.json`).
The registry at `http://192.168.0.15:4873` is the same Verdaccio as `http://localhost:4873`; anonymous reads are allowed, only publishing needs an `_authToken`. Verify a publish with `curl -s http://192.168.0.15:4873/@logger/cli | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>console.log('readme len:',(JSON.parse(s).readme||'').length))"`.
## Docker (collector server)
The `Dockerfile` builds an image that installs the published `@logger/collector` from Verdaccio and runs its `logging-collector` server entry:
```bash
docker build -t logger-collector:0.1.0 .
docker run -d --name logger-collector -p 4319:4319 -v logger-collector-data:/data logger-collector:0.1.0
```
Configure with env vars `PORT` (default `4319`), `HOST` (default `0.0.0.0`), `LOG_FILE` (default `/data/logs.jsonl`). Endpoints: `POST /ingest`, `GET /query`. Logs persist to the mounted volume.
The build reads from `http://host.docker.internal:4873` by default (Docker Desktop's alias for the host, where the Verdaccio port is published). Override with `--build-arg REGISTRY=http://192.168.0.15:4873` if your Docker network reaches the registry directly.
End-to-end (SDK writes, CLI reads), from a consumer project:
```bash
npm install @logger/sdk @logger/cli --registry=http://192.168.0.15:4873
# .logging/config.json -> { "server": { "baseUrl": "http://127.0.0.1:4319", "queryPath": "/query" }, "auth": {"type":"none"} }
npx log tail --remote --json --level=error
```
## Cloudflare Worker + D1 (remote collector)
`packages/worker` is the Cloudflare-native collector. It exposes the same HTTP surface as the Node collector, but stores records in D1:
- `GET /healthz` — unauthenticated liveness check.
- `POST /ingest` — authenticated ingest for one `LogRecord` or an array of records.
- `GET /query` — authenticated read-only tail query with `level`, `app`, `service`, `scope`, `since`, and `limit`.
The Worker uses D1 binding `DB` and Wrangler secret `LOGGING_TOKEN`. Do not put the token in `wrangler.jsonc`.
```bash
cd packages/worker
pnpm wrangler d1 create logger-collector
# copy the returned database_id into packages/worker/wrangler.jsonc
pnpm wrangler d1 migrations apply logger-collector --remote
pnpm wrangler secret put LOGGING_TOKEN
pnpm wrangler deploy
```
Consumer `.logging/config.json`:
```json
{
"version": 1,
"systemName": "shop",
"server": {
"baseUrl": "https://logger-collector.<account>.workers.dev",
"queryPath": "/query",
"ingestPath": "/ingest"
},
"auth": {"type": "bearer", "tokenEnv": "LOGGING_TOKEN"},
"defaults": {"app": "shop", "limit": 50},
"output": {"format": "compact", "redact": true, "maxLines": 200}
}
```
SDK usage:
```ts
import {createLogger, HttpTransport} from '@logger/sdk';
const log = createLogger({
app: 'shop',
scope: 'orders',
runtime: 'worker',
transport: new HttpTransport('https://logger-collector.<account>.workers.dev/ingest', {
headers: {authorization: `Bearer ${process.env.LOGGING_TOKEN}`},
}),
});
```
## Current Boundaries
Included:
- TypeScript SDK.
- Console, memory, batch, and HTTP transports.
- JSONL append-only store.
- HTTP ingest and read-only HTTP query endpoints.
- Cloudflare Worker remote collector backed by D1.
- CLI driven by `.logging/config.json`: `tail`, `config init`, `config doctor`, local and remote reads, redaction, and `maxLines` safety cap.
- `logging-agent` skill for Codex / Claude Code.
Not included yet:
- UI console.
- Multi-tenant access control (the Worker first version uses one shared Bearer token).
- Full-text search index.
- Alerting.
- Metrics and trace collection.
- OpenTelemetry exporter.