cfw-autumn/.plans/v2-cache-object-design.md

V2 Cache Object Design: Bounded FullCustomer + FullEntity

Decision: Flat entitlement structure

We use a flat entitlement-keyed cache format instead of the current nested customer_products[i].customer_entitlements[j] structure. This eliminates the path index Hash entirely and future-proofs for product-level splitting.

Current (nested):

$.customer_products[0].customer_entitlements[1].balance
  → requires path index HGET to resolve array indices

New (flat):

$.entitlements["cusEnt_abc"].balance
  → cusEntId IS the path, no index needed

Cache Format: CachedFullCustomer

Stored at {orgId}:env:fullcustomer:2.0.0:customerId

{
  "customer": {
    "id": "cus_123",
    "internal_id": "cus_abc",
    "org_id": "org_xyz",
    "env": "live",
    "name": "Acme",
    "email": "billing@acme.com",
    "fingerprint": "fp_...",
    "processor": { "id": "cus_stripe_123", "type": "stripe" },
    "processors": { ... },
    "metadata": { ... },
    "send_email_receipts": true,
    "auto_topups": [ ... ],
    "spend_limits": [ ... ],
    "usage_alerts": [ ... ],
    "overage_allowed": [ ... ],
    "created_at": 1700000000000
  },
  "products": {
    "cp_001": {
      "id": "cp_001",
      "internal_product_id": "prod_int_1",
      "product_id": "pro_plan",
      "internal_customer_id": "cus_abc",
      "internal_entity_id": null,
      "status": "active",
      "subscription_ids": ["sub_stripe_1"],
      "options": [ ... ],
      "created_at": 1700000000000,
      "product": { "id": "pro_plan", "name": "Pro Plan", ... },
      "customer_prices": [
        { "id": "cp_price_1", "price_id": "price_1", "price": { ... } }
      ],
      "free_trial": null
    }
  },
  "entitlements": {
    "ce_001": {
      "id": "ce_001",
      "internal_customer_id": "cus_abc",
      "internal_entity_id": null,
      "internal_feature_id": "feat_int_1",
      "customer_product_id": "cp_001",
      "entitlement_id": "ent_1",
      "balance": 950,
      "adjustment": 0,
      "additional_balance": 0,
      "unlimited": false,
      "usage_allowed": true,
      "next_reset_at": 1703000000000,
      "cache_version": 5,
      "entities": null,
      "entitlement": {
        "id": "ent_1",
        "feature": { "id": "api_calls", "internal_id": "feat_int_1", ... },
        "entity_feature_id": null,
        ...
      },
      "rollovers": [
        { "id": "ro_1", "balance": 100, "expires_at": 1705000000000, ... }
      ],
      "replaceables": []
    }
  },
  "extraEntitlements": {
    "ce_loose_1": {
      "id": "ce_loose_1",
      "customer_product_id": null,
      ...
    }
  },
  "entities": [
    { "id": "ety_1", "internal_id": "ety_int_1", "name": "Team A", ... }
  ],
  "subscriptions": [ ... ],
  "invoices": [ ... ],
  "aggregatedEntitlements": [ ... ],
  "aggregatedProducts": { ... },
  "aggregatedPrices": [ ... ]
}

Key design decisions

• products: keyed by cusProductId, NOT an array. Contains the product definition, prices, and free_trial — but NOT customer_entitlements (those are in entitlements).
• entitlements: keyed by cusEntId. Each entitlement has a customer_product_id field to link back to its product. Contains the full entitlement definition, rollovers, and replaceables inline.
• extraEntitlements: keyed by cusEntId, for loose entitlements where customer_product_id is null.
• entities: flat array (not keyed) since this is only used for entity lookup, not deduction. Only present on FullCustomer, not FullEntity.
• aggregated*: only on the bounded FullCustomer (customer-level view).

Cache Format: CachedFullEntity

Stored at {orgId}:env:fullentity:1.0.0:customerId:entityId

{
  "customer": {
    "id": "cus_123",
    "internal_id": "cus_abc",
    "processor": { ... },
    "spend_limits": [ ... ],
    "usage_alerts": [ ... ],
    "overage_allowed": [ ... ],
    "fingerprint": "fp_..."
  },
  "entity": {
    "id": "ety_1",
    "internal_id": "ety_int_1",
    "internal_customer_id": "cus_abc",
    "name": "Team A",
    "feature_id": "seats",
    "spend_limits": [ ... ],
    "usage_alerts": [ ... ],
    "overage_allowed": [ ... ]
  },
  "products": {
    "cp_entity_1": { ... },
    "cp_customer_level_1": { ... }
  },
  "entitlements": {
    "ce_entity_1": { "customer_product_id": "cp_entity_1", "balance": 500, ... },
    "ce_customer_level_1": { "customer_product_id": "cp_customer_level_1", "balance": 950, ... }
  },
  "extraEntitlements": {
    "ce_loose_1": { ... }
  }
}

What's included

• customer: subset of Customer fields needed by check/track (processor for Stripe lookup, billing controls, fingerprint for trial dedup)
• entity: the full entity record (billing controls, feature_id)
• products: entity-scoped products (internal_entity_id = this entity) + inherited customer-level products (internal_entity_id IS NULL). Keyed by cusProductId.
• entitlements: all entitlements from the included products. Keyed by cusEntId.
• extraEntitlements: loose entitlements matching this entity
• No entities array, aggregated*, subscriptions, invoices, trials_used

What's NOT included

• Other entities' products/entitlements
• Aggregated data (not needed for entity-level operations)
• Invoices, subscriptions (only needed for expand fields, fetched lazily)

Deduction Lua Script Changes

Current flow (path index)

1. HGET pathidx_key "cus_ent:{id}"    → { cp: 0, ce: 1 }
2. Build path: $.customer_products[0].customer_entitlements[1]
3. JSON.GET cache_key {path}           → entitlement object
4. JSON.NUMINCRBY cache_key {path}.balance -delta

New flow (flat, no path index)

1. Build path: $.entitlements["{cusEntId}"]
   (or $.extraEntitlements["{cusEntId}"] for loose entitlements)
2. JSON.GET cache_key {path}           → entitlement object
3. JSON.NUMINCRBY cache_key {path}.balance -delta

Changes to Lua scripts:

• find_entitlement_from_index → replaced with direct path construction: '$.entitlements["' .. cus_ent_id .. '"]'
• build_customer_entitlement_base_path / build_extra_customer_entitlement_base_path → replaced with single function that takes cusEntId and whether it's loose
• find_entitlement fallback (O(n) scan) → JSON.GET cache_key $.entitlements["{cusEntId}"] (O(1))
• Path index Hash is eliminated entirely — no HGET, no HSET, no separate Redis key to manage
• The entity_feature_id (previously stored in path index) needs to be on the entitlement object itself — it already is, via entitlement.entity_feature_id

Rollover paths:

• Current: $.customer_products[cp].customer_entitlements[ce].rollovers[i].balance
• New: $.entitlements["{cusEntId}"].rollovers[i].balance
• Rollover array index (i) is still needed but this is small (typically 0-3 rollovers)

Entity balance paths (old-style):

• Current: $.customer_products[cp].customer_entitlements[ce].entities["{entityId}"].balance
• New: $.entitlements["{cusEntId}"].entities["{entityId}"].balance

TypeScript Types

In-memory types (used by all endpoint logic)

// Bounded FullCustomer — no entity-scoped products
type BoundedFullCustomer = Customer & {
  customer_products: FullCusProduct[];     // internal_entity_id IS NULL only
  entities: Entity[];
  extra_customer_entitlements: FullCustomerEntitlement[];
  subscriptions?: Subscription[];
  invoices?: Invoice[];
  trials_used?: { product_id: string; customer_id: string; fingerprint: string }[];
  aggregated_customer_products?: FullCusProduct[];
  aggregated_customer_entitlements?: AggregatedCustomerEntitlement[];
  aggregated_customer_prices?: CustomerPrice[];
};

// FullEntity — entity-scoped + inherited customer-level products
type FullEntity = {
  customer: Customer;
  entity: Entity;
  customer_products: FullCusProduct[];     // entity + inherited
  extra_customer_entitlements: FullCustomerEntitlement[];
};

Note: BoundedFullCustomer has the same shape as the existing FullCustomer type (just fewer products). Existing code that operates on FullCustomer works unchanged. The FullEntity is a new type.

Cache serialization types

// What's stored in Redis (flat format)
type CachedCustomerDoc = {
  customer: Customer;
  products: Record<string, CachedProduct>;
  entitlements: Record<string, CachedEntitlement>;
  extraEntitlements: Record<string, CachedEntitlement>;
  entities?: Entity[];
  subscriptions?: Subscription[];
  invoices?: Invoice[];
  aggregatedEntitlements?: AggregatedCustomerEntitlement[];
  aggregatedProducts?: Record<string, CachedProduct>;
  aggregatedPrices?: CustomerPrice[];
};

type CachedEntityDoc = {
  customer: Customer;   // subset of fields
  entity: Entity;
  products: Record<string, CachedProduct>;
  entitlements: Record<string, CachedEntitlement>;
  extraEntitlements: Record<string, CachedEntitlement>;
};

type CachedProduct = Omit<FullCusProduct, 'customer_entitlements'>;

type CachedEntitlement = FullCustomerEntitlement & {
  customer_product_id: string | null;     // link back to product
};

Hydration: Cache → In-Memory

// CachedCustomerDoc → FullCustomer
const hydrateFullCustomer = (doc: CachedCustomerDoc): FullCustomer => {
  const entitlementsByProduct = groupBy(
    Object.values(doc.entitlements),
    (e) => e.customer_product_id
  );

  const customerProducts = Object.values(doc.products).map((product) => ({
    ...product,
    customer_entitlements: entitlementsByProduct[product.id] ?? [],
  }));

  return {
    ...doc.customer,
    customer_products: customerProducts,
    extra_customer_entitlements: Object.values(doc.extraEntitlements),
    entities: doc.entities ?? [],
    subscriptions: doc.subscriptions,
    invoices: doc.invoices,
    aggregated_customer_products: ...,
    aggregated_customer_entitlements: doc.aggregatedEntitlements,
    aggregated_customer_prices: doc.aggregatedPrices,
  };
};

// CachedEntityDoc → FullEntity
const hydrateFullEntity = (doc: CachedEntityDoc): FullEntity => {
  // Same groupBy pattern
  ...
};

Dehydration: In-Memory → Cache

// FullCustomer → CachedCustomerDoc
const dehydrateFullCustomer = (fullCustomer: FullCustomer): CachedCustomerDoc => {
  const products: Record<string, CachedProduct> = {};
  const entitlements: Record<string, CachedEntitlement> = {};

  for (const cusProduct of fullCustomer.customer_products) {
    const { customer_entitlements, ...productWithoutEnts } = cusProduct;
    products[cusProduct.id] = productWithoutEnts;

    for (const cusEnt of customer_entitlements) {
      entitlements[cusEnt.id] = { ...cusEnt, customer_product_id: cusProduct.id };
    }
  }

  const extraEntitlements: Record<string, CachedEntitlement> = {};
  for (const cusEnt of fullCustomer.extra_customer_entitlements) {
    extraEntitlements[cusEnt.id] = { ...cusEnt, customer_product_id: null };
  }

  return {
    customer: extractCustomerFields(fullCustomer),
    products,
    entitlements,
    extraEntitlements,
    entities: fullCustomer.entities,
    subscriptions: fullCustomer.subscriptions,
    invoices: fullCustomer.invoices,
    aggregatedEntitlements: fullCustomer.aggregated_customer_entitlements,
    ...
  };
};

What doesn't change

• All endpoint logic continues to use FullCustomer/FullEntity in-memory types
• getApiBalances, getApiSubscriptions, fullCustomerToCustomerEntitlements — all unchanged
• syncItemV3 — reads from cache, but the Lua deduction already writes cusEntId into sync messages, so sync just needs to know which cache key to read from
• Postgres schema — no changes
• V2 SQL query — no changes (returns flat rows, hydration produces FullCustomer)

What changes

• Cache set/get utilities — serialize to flat format, deserialize back
• Lua scripts — use direct $.entitlements["{id}"] paths instead of path index
• Path index — eliminated entirely (no separate Redis Hash key)
• buildPathIndex.ts — deleted
• fullCustomerCacheConfig.ts — new version + entity config
• Cache invalidation — entity-aware (see separate plan)

Migration path to product-level splitting (future)

If needed later, the flat structure makes this straightforward:

1. Move each product group (products[cpId] + its entitlements from entitlements where customer_product_id === cpId) to a separate Redis key
2. The Lua deduction script just receives a different cache key — the entitlement path $.entitlements["{cusEntId}"] stays identical within each product sub-doc
3. Full-read endpoints pipeline JSON.GET across product keys

No structural changes needed — just key routing.