Track token rate limit integration after AI #796

[nerdalert]

Purpose

Track the follow-on work needed after praxis-proxy/ai#796 establishes the initial token_rate_limit filter.

This is a cross-component integration tracker under the parent Token Rate Limiting epic. It does not move quota enforcement into Grid. Praxis AI enforces quota in the request path; Valkey provides shared state when gateways are horizontally scaled; Grid independently publishes provider-routing state.

Architecture boundary

authentication
  -> trusted request identity
  -> token_rate_limit
  -> intelligent_route
  -> selected provider gateway

• Authentication establishes a trusted principal.
• Praxis AI reserves and reconciles token quota.
• Valkey provides atomic shared quota state across gateway processes.
• Grid computes and distributes provider eligibility and selection metadata asynchronously.
• Praxis AI selects a provider from its in-memory routing snapshot only after quota admission succeeds.
• Grid and its operator must not participate in request-time quota checks.

Quota keys must remain independent of the selected provider, site, provider group, or routing revision. A route change must not create fresh quota.

Foundation supplied by AI #796

Capability	Status after #796	Notes
Sliding-window token limiting	Implemented	Enforces consumption over a rolling window.
Token-bucket limiting	Implemented	Supports burst capacity and sustained refill.
Per-rule algorithm selection	Implemented	Each rule selects sliding_window or token_bucket.
Ordered rules and optional header matching	Implemented	Rules are evaluated in configured order.
Pre-inference reservation	Implemented	Estimated tokens are reserved before provider work begins.
Post-response reconciliation	Implemented	Reservations are settled against reported token usage.
Incomplete-request handling	Implemented	Reservations have bounded timeout behavior.
In-memory backend	Implemented	Suitable for a single gateway process.
Shared Valkey backend	Implemented	Supports shared state and reuses multiplexed connections.
Hard-limit rejection	Implemented	Denied requests return HTTP 429.
Rate-limit response headers	Implemented	Token limit, remaining capacity, reset, and retry information are exposed as applicable.
Core unit and integration coverage	Implemented	Covers algorithms, backends, validation, reservation, and reconciliation behavior.

Follow-on capability gaps

Priority	Capability	Required outcome	Primary ownership	Tracking
P0	Trusted quota identity	Authentication publishes a verified, request-scoped principal that cannot be supplied or overridden by an untrusted client.	Praxis authentication/core	Parent: ai#121
P0	Principal and canonical-model quota keys	Shared quota is isolated by a stable identity/model contract rather than one budget shared by every request matching a rule.	Praxis AI	Related keying work: ai#129
P0	Pipeline ordering and lifecycle qualification	Authentication precedes quota admission, quota admission precedes routing/provider contact, and final token settlement executes exactly once for every terminal response path.	Praxis core and Praxis AI	ai#121
P0	Distributed enforcement qualification	Two or more gateway replicas enforce one atomic Valkey-backed quota through admission, denial, restart, expiry, and backend-failure scenarios.	Praxis AI and deployment integration	ai#121
P0	Routing independence qualification	Admitted requests may select different providers without changing quota identity; denied requests must stop before provider selection or contact.	Praxis AI integration	Provider-selection foundations: ai#731, grid#65
P1	Soft-cap or observe-only enforcement	Record an exceeded allocation without rejecting the request.	Praxis AI	New child issue needed
P1	Soft-to-hard policy changes	Allow an operator to move a rule between observation and enforcement with defined reload semantics.	Praxis AI	New child issue needed
P1	Configurable estimation	Configure the amount reserved before actual usage is known, with bounded and conservative behavior.	Praxis AI	Deferred by ai#796
P1	Token-type differentiation and weighting	Treat prompt/input, completion/output, reasoning, and total tokens according to an explicit policy.	Praxis AI	Deferred by ai#796
P1	Multiple budgets or tiers per rule	Enforce more than one window or allocation boundary atomically without partial reservations.	Praxis AI and Valkey backend	Deferred by ai#796
P1	Operational observability	Export bounded metrics and structured diagnostics for admissions, denials, reservations, reconciliation, expiry, backend failures, and active state.	Praxis AI	Deferred by ai#796
P2	Billing-grade metering	Define durable, auditable usage records separately from admission-control counters.	Praxis AI / metering integration	Deferred by ai#796
P2	Chargeback integration	Consume qualified metering records without coupling billing to Grid routing or the inference hot path.	External accounting integration	Requires metering contract first

Required integration behavior

• A verified principal receives the same quota regardless of which gateway replica accepts the request.
• A provider or routing change does not reset or partition the quota.
• Admission and reconciliation are atomic under concurrent requests.
• A hard denial returns 429 and does not contact a provider.
• Sliding-window capacity returns only as recorded usage ages out; recovery is not a global reset.
• Valkey failure behavior is explicit and testable, including whether the configured policy fails closed.
• Restarts do not silently discard shared quota state.
• Quota labels and telemetry remain bounded and do not expose credentials, raw authorization values, or unbounded user identifiers.
• Neither Grid nor an external metrics/control service is called during request-time quota admission.

Validation references

These experimental examples exercise useful end-to-end behavior while the upstream contracts evolve. They are validation references, not the issue structure or source of requirements:

• Distributed token rate limiting with Grid routing
• Intelligent overflow

The implementations and configuration shown there may change as the upstream proposal and component APIs are finalized.

Relationships

• Parent epic: praxis-proxy/ai#121
• Initial implementation: praxis-proxy/ai#796
• Per-key budget tracking: praxis-proxy/ai#129
• Praxis AI provider selection: praxis-proxy/ai#731
• Grid provider-selection contract: praxis-proxy/grid#65

Explicit follow-up tasks deferred from AI #796

These are accepted integration concerns, but they do not block the initial limiter foundation in AI #796. The proposal assumes identity has already been resolved upstream.

• Trusted matching and quota identity: consume authenticated, request-scoped identity metadata rather than relying on a client-controlled header as the trust source. Define spoofing protections and validate authentication-before-quota pipeline ordering.
• Probe and unrelated traffic scoping: ensure readiness probes, health checks, malformed requests, unauthenticated requests, and other non-inference traffic do not reserve token quota. Define either explicit request/path scoping or validated pipeline ordering before enabling a catch-all quota rule.