feat(metering): add external metering filter with balance checks and usage reporting

[noyitz]

Adds an external_metering HTTP filter: pre-request balance checks and
post-response token usage reporting against an external metering service.

Part of #577.

What the filter does

Request phase — resolves tenant identity, strips identity headers and
client credentials (authorization, x-api-key), and asks the metering
service whether the tenant may spend more tokens:

• GET {metering_url}/api/v1/customers/{username}/entitlements/{feature_key}/value?model={model}
• Exhausted budget → 429. Metering unreachable → admit when fail_open: true
  (default), 503 otherwise, so a metering outage degrades to unmetered service
  rather than an inference outage.

Response phase — emits a CloudEvents 1.0 event
(inference.tokens.used, or inference.request.error on upstream failure) to
POST {metering_url}/api/v1/events. Delivery is fire-and-forget: a slow or
failing metering service never delays a response the upstream already
answered. Token counts come from the token_count filter's filter_metadata
keys (token.input/output/total and the prompt cache breakdown from #582).

Identity resolution — three tiers, most trusted first

1. Verified metadata (unnamespaced {prefix}* keys) written by an
   authentication filter from verified credentials. When present, every lower
   tier is ignored entirely, so a client cannot spoof subscription or
   model via forged headers alongside valid credentials.
2. Guard metadata (namespaced identity.{prefix}* keys) written by the
   identity_header_guard filter (feat(filter): add identity header guard filter #709), which owns identity header capture.
3. Raw {prefix}* headers, for deployments where a trusted upstream auth
   layer (e.g. an external authorizer) injects them directly.

Identity headers are always stripped before the request reaches the upstream,
regardless of which tier supplied the identity.

Relationship to #709 and #577

This is the metering half of the split: #709 owns identity header capture into
namespaced metadata; this filter consumes that metadata (tier 2) and owns
admission control, credential stripping, and usage reporting. #577 documents
the original single-filter design and needs an update to reflect the split.

HTTP callouts use praxis-core's SubRequestClient (the CalloutClient named
in #577 was removed upstream in #849). register_ai_filters threads the shared
server-level client into the filter following the existing
openai_file_resolve pattern; without one the filter creates a private
connector, same as openai_file_resolve::from_config.
praxis_ai_apis::subrequest::execute_url is made public so filters can execute
full-URL sub-requests.

Size

1,876 insertions, of which 861 are unit + integration tests, 103 generated
docs + README rows, and 64 the example config. Functional filter code is
~1,010 lines across mod.rs, config.rs, and registration. Balance check,
event construction, and identity resolution are cohesive enough that splitting
them again would leave non-functional intermediate states; happy to split if
reviewers prefer.

Testing

• 41 unit tests (filters/src/metering/tests.rs), including spoofing-closure
  tests for the tier precedence rules
• 5 functional integration tests against examples/configs/external-metering.yaml
  (balance allowed, fail-closed rejection, header stripping, no-identity skip)
• make test — 4,905 passed, 0 failed; make lint green

[praxis-bot]

Review Summary

Clean, well-structured filter with good test coverage and correct conventions. Three medium-severity items found.

Severity	Count
Critical	0
Large	0
Medium	3

Findings:

1. [Medium] strip_client_credentials uses if let Ok for an infallible HeaderName parse, creating a silent-failure pattern for security-critical credential stripping. Use HeaderName::from_static("x-api-key") instead.
2. [Medium] validate_config only checks for empty prefix but does not validate that the prefix contains valid HTTP header name characters. An invalid prefix (e.g. containing spaces or control characters) would silently match no headers, leaving tenant identity headers unstripped.
3. [Medium] No test covers multi-value tenant headers (x-tenant-username: alice\r\nx-tenant-username: mallory). Add a unit test verifying all values are marked for removal when a client sends duplicate tenant headers.

[praxis-bot]

Review Summary

Clean, well-structured filter with good test coverage and correct conventions. The prior review covered three medium items; this pass adds one more.

Severity	Count
Critical	0
Large	0
Medium	1

Findings:

1. [Medium] strip_identity_headers performs two unnecessary per-request allocations: lowering HeaderName::as_str() (already lowercase by http crate contract) and lowering the immutable prefix on every call instead of once at construction.

[praxis-bot]

No new findings beyond prior review.

[praxis-bot]

No new findings beyond prior reviews.

Fourth pass confirms the four existing findings (infallible HeaderName parse, prefix validation gap, missing multi-value header test, redundant to_ascii_lowercase) are the substantive items. Code structure, conventions, test coverage, integration tests, documentation, and registry wiring are all correct.

[praxis-bot]

Review Summary

Fifth pass. One new medium finding (assertion messages in unit tests). Four items from prior reviews remain unaddressed in the code.

Severity	Count (new)	Count (prior, open)
Critical	0	0
Large	0	0
Medium	1	4

New finding:

1. [Medium] Unit test assertions missing messages (tests.rs). All 15 assert!/assert_eq! calls lack a message string. The workspace enforces missing_assert_message = "deny" (Cargo.toml line 223), and the test module's #[allow] only covers unwrap_used and expect_used. The integration tests correctly include messages on every assertion.

Open prior findings (no code changes since last review):

1. HeaderName::from_static("x-api-key") instead of fallible .parse() with if let Ok (mod.rs:102)
2. Prefix validation for valid HTTP header characters (config.rs:30)
3. Missing multi-value tenant header test (tests.rs)
4. Redundant per-request to_ascii_lowercase on both key and prefix (mod.rs:88)

[leseb]

@noyitz please resolve the conflicts and address the bot's comments thanks

[jordigilh]

A few things flagged inline, plus some broader ones here. Identity stripping, the CloudEvents payload, and URL-encoding of client-influenced path segments all held up -- no SSRF or credential-leakage concerns.

Circuit breaking is available but unused for this gateway. SubRequestConnector supports one (with_options, praxis#879) and the base proxy wires it up; praxis-ai's server never does (server.rs#L141, commands.rs#L40 both use plain ::new()). runtime.subrequest_circuit_breaker parses fine in an AI-gateway config today and is silently dropped. external_metering is the first mandatory-every-request consumer of that shared pool (the other 5 are opportunistic) -- worth a follow-up before a high-traffic rollout.

execute_url (+ client/error types) are now pub, not pub(crate) -- needed cross-crate, but worth flagging as a public API addition on praxis-ai-apis, semver-relevant going forward.

Generated docs skip the 3-tier identity design -- it only lives in a doc comment on the private read_identity_headers, which the doc generator never harvests (it reads a second paragraph of the module/struct doc -- credential_inject.md already does this). Worth moving it up.

#577 (this filter's linked design doc) still describes the single-filter/CalloutClient design this PR (split across #581 + #709) supersedes.

Minor: iterative_request_router's validate() doesn't block nesting external_metering inside a step (it does block nested IRR and compression) -- would double-report with no idempotency key if misconfigured. Not urgent, just flagging.

[noyitz]

@jordigilh thanks for the thorough pass — all six inline findings are addressed in 79c3709 (replied on each thread), make test 4,913 green, lint green. On the broader notes:

• Circuit breaker: agreed this filter is the first mandatory-per-request consumer of the shared pool. Wiring SubRequestConnector::with_options through the server (so runtime.subrequest_circuit_breaker stops being silently dropped) deserves its own PR — happy to open the follow-up issue.
• execute_url going public: acknowledged as a deliberate API addition on praxis-ai-apis; the alternative was duplicating the URL/DNS/peer-fallback logic in the filters crate.
• Docs: the three-tier identity design now lives in the struct documentation, so the generated external_metering.md carries it (regenerated in 79c3709).
• feat(filter): add external metering filter for usage reporting and balance checks #577: updating it now to reflect the feat(metering): add external metering filter with balance checks and usage reporting #581 + feat(filter): add identity header guard filter #709 split and the CalloutClient → SubRequestClient migration.
• IRR nesting: agreed, will fold an external_metering guard into the same follow-up as the ordering-validation idea.

[praxis-bot]

Re-review (6th pass)

All four code findings from prior reviews have been addressed:

• HeaderName::from_static("x-api-key") (fixed, line 667)
• Prefix validation for valid header characters (fixed, config.rs lines 99–106)
• Multi-value identity header test (added, tests.rs lines 411–426)
• Redundant per-request to_ascii_lowercase (pre-lowered at construction, line 229)

Assertion messages (R5) are partially addressed: security and spoofing tests now include messages, config/parsing tests still lack them.

One new medium finding below.

Severity	Count (new)
Critical	0
Large	0
Medium	1

[jordigilh]

Approving — all six findings resolved: balance-check contract documented, tier-gating fixed, configurable metadata namespace added, failure-reporting metric added, nested-JSON model extraction hardened (79c3709), and the cross-filter ordering-constraint gap now has a tracked follow-up (ai#812).

[leseb]

mostly stamping since @jordigilh did some thorough reviews already, thanks!

[praxis-bot]

PR Re-review

Re-review: new commits since last review on 2026-08-20.

The force-pushed commits address four of the six prior findings (from_static for x-api-key, prefix validation, multi-value header test, pre-lowered prefix). The query-string encoding gap and partial assertion-message gap remain open from prior reviews and are not repeated here.

Summary: One new medium finding — the default_username and default_model fallback paths lack runtime test coverage.

Severity	Count
Critical	0
Large	0
Medium	1

[leseb]

@noyitz see ci failures