Skip to content

Make delayed AI costs converge to one canonical ledger #118

Description

@farzanmrz

Problem

Delayed gateway pricing can repair model_calls.cost_usd, but the matching usage_events.cost_usd remains whatever was available at write time—usually null or zero for BYOK calls. The two ledgers therefore permanently disagree after reconciliation.

reconcileMissingCosts() has the correct model_calls.generation_id, but usage_events has no model_call_id. Its only apparent correlation key, (ref_id, kind), is not one-to-one: multiple drafting-council calls for the same source post share that pair. A guessed join would assign costs to the wrong event.

Production reconciliation is also coupled to the end of a later ingest delivery and can be skipped when the request's 800-second budget is nearly exhausted. Even when that sweep succeeds, it repairs only model_calls.

Impact

COGS, usage metering, margin analysis, and any plan or alert built from usage_events can understate AI spend while model_calls reports a different total. We do not currently have one canonical cost ledger that reliably converges after the gateway publishes delayed pricing.

Evidence

  • lib/agent/gateway-cost.ts:149-164 documents the known permanent divergence and the missing one-to-one relationship.
  • lib/agent/draft-pipeline.ts:229 stamps usage_events independently with the cost known at write time.
  • app/api/ingest/route.ts:89-115 runs the only production reconciliation from post-response work and may skip it when the request budget is depleted.
  • supabase/migrations/20260722044255_experiments_voice_schema.sql:107-119 defines usage_events without a model-call relationship.

Acceptance

  • Designate one canonical source of truth for per-model-call cost and document which ledger downstream COGS and plan calculations must use.
  • Add a reliable relationship such as usage_events.model_call_id, or remove duplicate per-call cost storage so an invalid divergent state cannot be represented.
  • Do not correlate rows by (ref_id, kind); multiple council calls sharing that pair must remain distinguishable.
  • Make delayed pricing converge idempotently without requiring another customer delivery to arrive.
  • Define a safe migration/backfill strategy for existing rows, explicitly handling historical rows that cannot be matched without guessing.
  • Test delayed BYOK pricing, multiple calls sharing one source reference, retries, genuinely free/zero-cost calls, and repeated reconciliation.
  • Surface or log unresolved cost rows so permanently unpriced spend is observable.

Out of scope

Provider selection, pricing changes, subscription-plan design, and unrelated usage-event kinds are not part of this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions