cfw-logging/README.md

Logger System

Independent TypeScript logger collection system with:

• @logger/sdk: application logger API and transports.
• @logger/collector: JSONL store, HTTP ingest, and read-only HTTP query.
• @logger/cli: log tail / log config for reading records, driven by a .logging/config.json profile.
• logging-agent skill: a usage contract for Codex / Claude Code to read logs safely.

Install

pnpm install
pnpm test
pnpm build

SDK

import {createLogger, HttpTransport} from '@logger/sdk';

const log = createLogger({
  app: 'shop',
  service: 'orders',
  scope: 'orders.refund',
  runtime: 'server',
  transport: new HttpTransport('http://127.0.0.1:4319/ingest'),
});

log.info('hello');
log.error('error msg', new Error('payment timeout'), {orderId: 'o-2099'});
await log.flush();

Records are structured JSON with timestamp, level, message, app, service, scope, runtime, optional attributes, optional serialized error, and version: 1.

Sensitive fields are redacted by default when passed in attributes:

• password
• token
• secret
• apiKey
• authorization
• cookie
• set-cookie

Collector

Use JsonlLogStore directly:

import {JsonlLogStore} from '@logger/collector';

const store = new JsonlLogStore('.logging/logs.jsonl');
await store.append(record);
const latest = await store.tail({limit: 10, level: 'error'});

Start an HTTP collector server from application code:

import {createHttpCollectorServer, JsonlLogStore} from '@logger/collector';

const store = new JsonlLogStore('.logging/logs.jsonl');
createHttpCollectorServer(store).listen(4319);

Endpoints:

• POST /ingest — append a single LogRecord or an array of LogRecord objects.
• GET /query — read-only tail. Supports query params level, app, service, scope, since, limit; returns a JSON array of matching records (last limit, file order).

Configuration (.logging/config.json)

The CLI is driven by a project-local profile at .logging/config.json. It declares the system name, where logs live, optional remote access, and safe defaults. Secrets are never stored in the file — only the name of the environment variable that holds a token.

{
  "version": 1,
  "systemName": "shop",
  "local": {"file": ".logging/logs.jsonl"},
  "server": {"baseUrl": "https://logs.example.com", "queryPath": "/query", "ingestPath": "/ingest"},
  "auth": {"type": "bearer", "tokenEnv": "LOGGING_TOKEN"},
  "defaults": {"app": "shop", "service": "orders", "limit": 50},
  "output": {"format": "compact", "redact": true, "maxLines": 200}
}

Required: version (must be 1) and systemName. Everything else is optional. See skills/logging-agent/references/config-contract.md for the full field reference and security rules.

Bootstrapping a profile

log config init --system=shop
log config init --system=shop --server=https://logs.example.com --token-env=LOGGING_TOKEN

config init writes .logging/config.json plus a .logging/.gitignore that ignores *.local.json, *.secret.json, tokens*, and logs.jsonl while keeping config.json committable. Pass --force to overwrite an existing profile.

Checking a profile

log config doctor

Validates the config shape, confirms systemName, checks that any required token environment variable is set (without printing its value), probes remote reachability when a server is configured, and checks the local file. Exits non-zero if any hard check fails.

CLI

After build:

node packages/cli/dist/index.js tail
node packages/cli/dist/index.js tail --level=error --app=shop --json
node packages/cli/dist/index.js tail --remote --json

The package binary name is log:

log tail
log tail --level=error --app=shop --service=orders --scope=orders.refund --json
log config init --system=shop
log config doctor

Where tail reads from

tail resolves its source predictably and never silently calls a remote service:

1. --remote → query the configured server (requires server.baseUrl; auth resolved from auth).
2. --file=<path> or LOGGER_FILE → read that local JSONL file.
3. config.local.file → read the configured local file.
4. config.server configured with no local source → query the server.
5. fallback → read .logging/logs.jsonl and print a hint to run log config init.

Filter and output resolution

Command-line flags override config.defaults, which override the built-in defaults. Supported filters: --level, --app, --service, --scope, --since, -n/--limit. Output is capped to config.output.maxLines (default 200); --json emits one record per line; output.redact (default true) masks known sensitive fields as a safety net even for records that were not redacted at ingest.

Missing config, missing token, or a missing query endpoint each produce a clear error message instead of a silent guess.

Publishing (private Verdaccio)

Build first, then publish with the helper script:

pnpm build
./scripts/publish.sh            # version defaults to packages/shared-schema's version
./scripts/publish.sh 0.2.2      # or pass an explicit version

Use the script instead of pnpm publish -r. Verdaccio v6 does not extract README from the tarball — it only reads the manifest readme field, which modern npm/pnpm publish no longer populate, so a plain publish makes the registry UI show "No README data found!". The script injects readme into each published manifest (rewriting workspace:* to the real version on the way, then restoring package.json).

The registry at http://192.168.0.15:4873 is the same Verdaccio as http://localhost:4873; anonymous reads are allowed, only publishing needs an _authToken. Verify a publish with curl -s http://192.168.0.15:4873/@logger/cli | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>console.log('readme len:',(JSON.parse(s).readme||'').length))".

Docker (collector server)

The Dockerfile builds an image that installs the published @logger/collector from Verdaccio and runs its logging-collector server entry:

docker build -t logger-collector:0.1.0 .
docker run -d --name logger-collector -p 4319:4319 -v logger-collector-data:/data logger-collector:0.1.0

Configure with env vars PORT (default 4319), HOST (default 0.0.0.0), LOG_FILE (default /data/logs.jsonl). Endpoints: POST /ingest, GET /query. Logs persist to the mounted volume.

The build reads from http://host.docker.internal:4873 by default (Docker Desktop's alias for the host, where the Verdaccio port is published). Override with --build-arg REGISTRY=http://192.168.0.15:4873 if your Docker network reaches the registry directly.

End-to-end (SDK writes, CLI reads), from a consumer project:

npm install @logger/sdk @logger/cli --registry=http://192.168.0.15:4873
# .logging/config.json -> { "server": { "baseUrl": "http://127.0.0.1:4319", "queryPath": "/query" }, "auth": {"type":"none"} }
npx log tail --remote --json --level=error

Cloudflare Worker + D1 (remote collector)

packages/worker is the Cloudflare-native collector. It exposes the same HTTP surface as the Node collector, but stores records in D1:

• GET /healthz — unauthenticated liveness check.
• POST /ingest — authenticated ingest for one LogRecord or an array of records.
• GET /query — authenticated read-only tail query with level, app, service, scope, since, and limit.

The Worker uses D1 binding DB and Wrangler secret LOGGING_TOKEN. Do not put the token in wrangler.jsonc.

cd packages/worker
pnpm wrangler d1 create logger-collector
# copy the returned database_id into packages/worker/wrangler.jsonc
pnpm wrangler d1 migrations apply logger-collector --remote
pnpm wrangler secret put LOGGING_TOKEN
pnpm wrangler deploy

Consumer .logging/config.json:

{
  "version": 1,
  "systemName": "shop",
  "server": {
    "baseUrl": "https://logger-collector.<account>.workers.dev",
    "queryPath": "/query",
    "ingestPath": "/ingest"
  },
  "auth": {"type": "bearer", "tokenEnv": "LOGGING_TOKEN"},
  "defaults": {"app": "shop", "limit": 50},
  "output": {"format": "compact", "redact": true, "maxLines": 200}
}

SDK usage:

import {createLogger, HttpTransport} from '@logger/sdk';

const log = createLogger({
  app: 'shop',
  scope: 'orders',
  runtime: 'worker',
  transport: new HttpTransport('https://logger-collector.<account>.workers.dev/ingest', {
    headers: {authorization: `Bearer ${process.env.LOGGING_TOKEN}`},
  }),
});

Current Boundaries

Included:

• TypeScript SDK.
• Console, memory, batch, and HTTP transports.
• JSONL append-only store.
• HTTP ingest and read-only HTTP query endpoints.
• Cloudflare Worker remote collector backed by D1.
• CLI driven by .logging/config.json: tail, config init, config doctor, local and remote reads, redaction, and maxLines safety cap.
• logging-agent skill for Codex / Claude Code.

Not included yet:

• UI console.
• Multi-tenant access control (the Worker first version uses one shared Bearer token).
• Full-text search index.
• Alerting.
• Metrics and trace collection.
• OpenTelemetry exporter.