Skip to content

feat(compact): reactive compaction with direct input support - #726

Open
crstrn13 wants to merge 651 commits into
praxis-proxy:mainfrom
crstrn13:feat/compact-filter-issue-30
Open

feat(compact): reactive compaction with direct input support#726
crstrn13 wants to merge 651 commits into
praxis-proxy:mainfrom
crstrn13:feat/compact-filter-issue-30

Conversation

@crstrn13

@crstrn13 crstrn13 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Completes the remaining work items from #30:

  • Use previous_usage for token estimationshould_compact() checks the stored usage.total_tokens from the rehydrated response before falling back to local tiktoken counting
  • Configurable summary prefixsummary_prefix config option (default: [Previous conversation summary]) carried through to compaction items and both translation paths
  • Hide compaction items from input_items APInormalize_input_items() filters out {"type": "compaction"} items so clients never see internal state
  • Persist compaction as a stored response — after summarization, the compacted response is written back to the response store with the compaction item in its input
  • Explicit compact endpointPOST /v1/responses/{id}/compact triggers compaction on a previously stored response

Test plan

  • cargo test -p praxis-ai-apis — new unit tests for all 5 items
  • cargo test -p praxis-ai-filters — no regressions in filter tests
  • make lint passes
  • Manual test with example config against a real inference backend

leseb and others added 30 commits July 8, 2026 08:40
…axis-proxy#308)

The lint-filter-docs check fails in CI because parse_shared_config_items()
hardcodes ../praxis to find shared config types. Since praxis moved from
a path dependency to a crates.io dependency (v0.4.0), ../praxis does not
exist in CI.

Fall back to resolving praxis-proxy-filter source via cargo metadata when
../praxis is not available. Also trim unused core/tls source parsing since
only filter/payload_processing types (OnInvalidBehavior) are referenced by
AI filter configs.

Signed-off-by: Sébastien Han <seb@redhat.com>
…dencies group (praxis-proxy#307)

chore(deps): bump jsonwebtoken in the rust-dependencies group

Bumps the rust-dependencies group with 1 update: [jsonwebtoken](https://github.com/Keats/jsonwebtoken).


Updates `jsonwebtoken` from 9.3.1 to 10.4.0
- [Changelog](https://github.com/Keats/jsonwebtoken/blob/master/CHANGELOG.md)
- [Commits](Keats/jsonwebtoken@v9.3.1...v10.4.0)

---
updated-dependencies:
- dependency-name: jsonwebtoken
  dependency-version: 10.4.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: rust-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
…roxy#306)

Bumps [actions/github-script](https://github.com/actions/github-script) from 7.0.1 to 9.0.0.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](actions/github-script@60a0d83...3a2844b)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
…axis-proxy#305)

Bumps [github/codeql-action/init](https://github.com/github/codeql-action) from 4.36.2 to 4.36.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@8aad20d...54f647b)

---
updated-dependencies:
- dependency-name: github/codeql-action/init
  dependency-version: 4.36.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#303)

Bumps [github/codeql-action/analyze](https://github.com/github/codeql-action) from 4.36.2 to 4.36.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@8aad20d...54f647b)

---
updated-dependencies:
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.36.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
…proxy#304)

Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 6.1.0 to 6.2.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](docker/metadata-action@80c7e94...dc80280)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: 6.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
…xy#302)

Bumps [docker/login-action](https://github.com/docker/login-action) from 4.2.0 to 4.4.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@650006c...af1e73f)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
…-proxy#248)

* feat(conversations): rehydrate and append via conversation.id

When a POST /v1/responses request includes a `conversation` field
(string ID or `{"id": "..."}` object), load the stored conversation
and prepend its messages to the input. `previous_response_id` takes
precedence when both are present.

After a successful non-streaming response with status "completed",
input and output items are appended back to the conversation so
subsequent requests see the full accumulated history.

- Rehydrate filter: add conversation-based rehydration path alongside
  existing previous_response_id support
- Proxy filter: strip `conversation` field from outbound body
- Conversations filter: add response-phase append-back with guards
  for streaming, background, and non-completed responses
- Store filter: trigger store init when has_conversation metadata set
- Full-flow config: add openai_conversations filter with
  on_invalid: continue for format classifier compatibility
- Add unit and integration tests for all new paths

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(tests): remove stale ai-inference feature gate from conversations_rehydrate

The ai-inference feature no longer exists in the integration test crate.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test(conversations): add unit tests for append-back response hooks

Cover on_response arming logic (metadata guards, content-type, status
code) and on_response_body paths (skip non-completed, invalid JSON,
empty items, full persist cycle) to restore coverage above 90%.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(lint): resolve clippy and compile errors after rebase

Suppress large_stack_frames on on_request dispatcher and fix
BackendGuard import path after test-utils restructuring on main.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(config): restore original backend endpoint port in full-flow example

The full-flow example config endpoint was changed from 3001 to 8000,
breaking the integration tests that use port mapping keyed on
127.0.0.1:3001.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix: address review feedback on conversation rehydration

- Reject malformed conversation values (e.g. {}, 42) with 400 instead
  of silently forwarding them to the backend
- Defensively strip conversation from proxy passthrough body as a
  safety net against unresolved fields leaking upstream
- Propagate append-back store errors with ? instead of swallowing
  them, consistent with the response store filter
- Replace fixed sleep in integration test with retry/poll loop
- Add tenant metadata mismatch test and make mock store tenant-aware

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix: revert append-back to warn+swallow and add conversation store failure test

Append-back is a best-effort side-effect — propagating store errors
would abort the client's already-successful inference response. Revert
to warn and continue.

Add missing test for store failure during conversation rehydration
(the existing test only covered the previous_response_id path).

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
)

* test(mcp): add unit tests for broker config validation (praxis-proxy#278)

Add 63 inline unit tests to config.rs covering CacheScope,
InvalidToolPolicy, serde deserialization, validate_schema_object
edge cases, validate_path, server/tool/exposed name uniqueness,
cache field profile validation, version validation, build_config
integration, and catalog entry construction.

Also fix pre-existing unfulfilled #[expect(dead_code)] on
CatalogTool by using cfg_attr(not(test), expect(...)) since test
builds use all fields.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test(mcp): add coverage for broker config validation gaps

Add tests for uncovered code paths in MCP broker config validation:

- build_config_minimal: assert default_version and supported_versions
  are correctly derived from the current protocol profile
- validate_server_paths: direct unit tests with invalid path and
  multi-server iteration proving the second server is checked
- validate_server_clusters: positive test with valid non-empty cluster
- build_config_with_tools_and_prefix: assert input_schema preservation,
  cluster, backend_path, and server_name on catalog entries

Widen validate_server_paths and validate_server_clusters to pub(super)
so the broker test module can call them directly.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test(mcp): address review feedback on broker config tests

Remove short-form separator comments from the test module — test
function names provide sufficient organization.

Replace fragile Debug-format assertions with direct field access for
cluster and backend_path on catalog entries, and remove the associated
inline comment.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
* test: replay sanitized agent sessions

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* chore: update crossbeam-epoch advisory fix

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* docs: explain replay fixture examples

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: share temp sqlite helper

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: address replay review comments

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: add session replay importer tooling

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: assert replayed response body is stored

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: harden replay fixture paths

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: avoid replay fixture source path leaks

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

---------

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
test(a2a): add unit tests for envelope parsing (praxis-proxy#280)

Signed-off-by: Sébastien Han <seb@redhat.com>
…tion (praxis-proxy#309)

* feat(filter): add tool_parse filter for Responses API tool classification

Add a new tool_parse filter that parses the tools array and tool_choice
from Responses API POST /v1/responses requests. Classifies each tool by
type (function, web_search, file_search, code_interpreter, computer_use,
image_generation, tool_search, MCP) and promotes summary facts to
metadata and filter results for branch conditions.

The filter does not mutate the request body and is gated to only process
Responses API create requests. Malformed function tools (missing name)
are still counted by discriminator for has_tools routing. Unsafe values
(oversized, control characters) in tool_choice are silently dropped
rather than failing the filter.

Closes praxis-proxy#42

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(tool_parse): promote function_count to filter results

function_count was written to metadata but never promoted to
filter_results or restored, making it invisible to branch_chains
conditions. Add it to promote_filter_results() and restore via
restore_function_count().

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(tool_parse): recognize web_search_2025_08_26 as WebSearch

The OpenAI API lists web_search_2025_08_26 as a valid web search tool
type. Without this match arm, requests using that literal fall through
to Unknown, setting has_tools=true but has_web_search=false, causing
branch_chains on has_web_search to silently miss them.

Signed-off-by: Sébastien Han <seb@redhat.com>

* docs(tool_parse): note why accumulate_tool extracts full details

The parser clones names, configs, and MCP entries that tool_parse
does not consume today. Document that tool_dispatch (praxis-proxy#26) will need
these fields.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test(tool_parse): assert FilterAction and add restore unit tests

Assert Release on has-tools paths and Continue on no-tools/skip paths
in every run_filter caller. Add unit tests for restore_presence_flags,
restore_function_count, and restore_tool_choice (present and absent).

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
praxis-proxy#261)

feat(apis): add Deserialize to TokenUsageProvider and streaming token extraction

  Add serde Deserialize to TokenUsageProvider for YAML config parsing.
  Add extract_streaming_tokens for providers that spread token counts
  across multiple SSE events (Anthropic message_start/message_delta,
  Bedrock ConverseStream metadata).

Signed-off-by: szedan <szedan@redhat.com>
test(config): add unit tests for remaining config modules (praxis-proxy#283)

Add inline #[cfg(test)] modules to the four config files that had
zero test coverage:

- filters/src/prompt_enrich/config.rs (17 tests)
- apis/src/openai/responses/model_rewrite/config.rs (24 tests)
- apis/src/openai/responses/config.rs (9 tests)
- apis/src/anthropic/messages_format/config.rs (10 tests)

Tests cover serde defaults, deny_unknown_fields enforcement,
validation functions, null-header-disables-promotion, and role/enum
round-trips.

Signed-off-by: Sébastien Han <seb@redhat.com>
…s-proxy#317)

* docs: use language-neutral type names in generated filter docs

Replace Rust-specific type names (u8, u16, u32, u64, usize, i32, f64,
etc.) with language-neutral equivalents (integer, number) in the xtask
filter-doc generator and regenerate all filter documentation.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test(xtask): table-driven coverage for all numeric primitives

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
…is-proxy#262)

feat(filter): add token_count filter for streaming and non-streaming responses

  Implements issue 211. Extracts token usage from AI inference responses
  across all five providers via content-type detection. Reuses the SSE
  scanner from A2A and the token extraction APIs. Writes unified counts
  to filter_metadata for downstream consumers like token_usage_headers.

Signed-off-by: szedan <szedan@redhat.com>
test(a2a): add unit tests for config validation (praxis-proxy#279)

Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
)

test(token_usage): add unit tests for provider parsers (praxis-proxy#281)

Add 26 inline tests to providers.rs covering edge cases not in
the existing tests.rs: missing/null fields, zero values, cache
token permutations, saturating_add overflow, Bedrock Converse
fallback paths, and malformed inputs across all four parsers.

Signed-off-by: Sébastien Han <seb@redhat.com>
* Preserve source records in session replay fixtures

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* Fix session replay importer lint

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* Cover image replay chat completions translation

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* Address session replay review feedback

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* docs: add anthropic messages replay test plan

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: guard replay fixtures against local paths

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: normalize replay fixture source label

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: normalize replay fixture source metadata

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: preserve redacted replay id relationships

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: keep replay fixture hygiene helper focused

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: ignore redacted thinking in replay import

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: accept string assistant replay content

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* test: move claude assistant source records

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

---------

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
)

test(guardrails): add unit tests for filter and config (praxis-proxy#282)

Add config-level unit tests for ProviderType serde, PhaseConfig
defaults/overrides/deny_unknown_fields, and AiGuardrailsConfig
serde validation. Strengthen the body_mode test to assert the exact
1 MiB StreamBuffer limit. Convert dead_code expects on PhaseConfig
fields to cfg_attr(not(test)) so tests can read the fields.

Signed-off-by: Sébastien Han <seb@redhat.com>
Adds local A2A context-owner routing on top of existing task routing.
When Praxis observes a backend create or update a task with contextId,
it records the context owner and routes later ListTasks / SendMessage /
SendStreamingMessage requests for that same context back to the owning
backend. Task ID routes still take precedence over context routes.
Context routes always use ttl_seconds; a completed task does not evict
its context route, because later messages or ListTasks calls in the same
context still need affinity. This is local in-process state only;
Redis/Valkey multi-replica state remains follow-up work.

Signed-off-by: Brent Salisbury <brent.salisbury@gmail.com>
* test: add unit tests to boost coverage to 95%

Add unit tests across 12 files to increase line coverage from ~94.5%
to 95.35%, covering previously untested code paths: error handling,
edge cases, Display impls, and pure helper functions.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(test): use CWD_MUTEX + CwdGuard RAII pattern in commands tests

Addresses review feedback: replace raw set_current_dir with the
CwdGuard pattern from watcher.rs to prevent cwd leaks on panic
and eliminate race windows with parallel tests.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
…raxis-proxy#240)

feat(token_count): add Bedrock InvokeModel header-based extraction (praxis-proxy#210)

PR praxis-proxy#262 merged an equivalent token_count filter to main while this
branch was in flight. Rebase onto it and extend it to close the one
gap against the praxis-proxy#210 proposal: Bedrock InvokeModel returns token
counts as response headers (x-amzn-bedrock-input/output-token-count)
rather than in the body, so it needs a distinct extraction path.

Introduces a local ProviderKind config enum (OpenAi, Anthropic,
Google, Bedrock, BedrockInvokeModel, Azure) that maps to the shared
TokenUsageProvider for body-based providers and short-circuits to
header-only extraction for BedrockInvokeModel, with BodyAccess::None
so no body is ever buffered for that provider.

Also fixes a backwards ordering note in the token-usage-headers
example: response hooks run in reverse declared order, so a
token-counting filter must be declared after token_usage_headers,
not before.

Signed-off-by: mkoushni <mkoushni@redhat.com>
…xis-proxy#239)

docs(proposals): add streaming token accumulation proposal for praxis-proxy#119 for incremental SSE token counting so maintainers can review goals and motivation before design.

Signed-off-by: Hen Schwartz <hschwart@redhat.com>
rkaplan-hub and others added 2 commits August 24, 2026 23:04
…raxis-proxy#702)

* refactor(responses): centralize usage accumulation

Signed-off-by: kaplan <rivka.kaplan@nokia.com>
Signed-off-by: kaplan <rkaplan@redhat.com>

* style: apply rustfmt to responses usage modules

Signed-off-by: kaplan <rkaplan@redhat.com>

---------

Signed-off-by: kaplan <rivka.kaplan@nokia.com>
Signed-off-by: kaplan <rkaplan@redhat.com>
…ing usage (praxis-proxy#782)

* fix(token_usage): recover from SSE overflow instead of silently dropping usage

Token accounting stopped permanently after a single oversized SSE event
or JSON response, clearing all working state (including the terminal
usage event) with no signal that data was lost.

- The shared SSE scanner (used by token_usage and a2a) now discards only
  the oversized event and resumes at the next event boundary, instead of
  aborting the whole stream. A terminal usage event arriving after an
  oversized one is now captured.
- JSON and any residual SSE overflow (e.g. the usage event itself being
  oversized) now set an explicit token.status=overflow metadata key and
  Praxis-Token-Status response header, so billing consumers cannot
  mistake missing counts for zero usage.
- max_body_bytes/max_scratch_bytes are now configurable per token_count
  filter instance instead of fixed at 1 MiB / 64 KiB.

Fixes praxis-proxy#674

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(token_usage): share SkipPhase encoding and reject zero capture limits

Give A2A and token_count a single SkipPhase metadata codec, reject
zero max_body_bytes/max_scratch_bytes, and regenerate filter docs so
lint CI matches the new config fields.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(token_usage): mark overflow when a dropped SSE event is the tail

Partial Anthropic/Bedrock counts followed by an oversized terminal
usage event now keep the captured maxima and set token.status=overflow,
while a recovered usage event after a drop stays authoritative.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(token_usage): cap max_scratch_bytes at the shared 64 MiB ceiling

Signed-off-by: mkoushni <mkoushni@redhat.com>

---------

Signed-off-by: mkoushni <mkoushni@redhat.com>

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review (Aug 24 commits)

The two new commits fix the summarization scope (exclude current input from summarization text in the rehydrated path), add is_compactable unit tests, add an explicit compact endpoint integration test, fix the stale docstrings on CompactFilter and the docs page, and add store initialization for the compact endpoint.

Previous findings addressed:

  • Stale docstrings: fixed (struct doc and docs page updated)
  • Missing is_compactable / ensure_compactable_state tests: fixed (extracted is_compactable and added 5 tests)

One new finding below.

Severity Count
Medium 1

Comment thread apis/src/openai/responses/store/filter.rs Outdated
Signed-off-by: Alexander Cristurean <acristur@redhat.com>
…and store filters

Signed-off-by: Alexander Cristurean <acristur@redhat.com>
aslakknutsen and others added 19 commits August 26, 2026 10:12
fix: add missing praxis-main API fields

Praxis main added session_stores and pinned_endpoint_address
to HttpFilterContext, log_level to AdminEndpointOptions, and
a log_level argument to run_server.

Quote on_result.result as "true" so YAML does not parse it
as a boolean.

Signed-off-by: Aslak Knutsen <aslak@4fs.no>
…y#815)

* feat(azure): add azure_ad upstream authentication filter

Add an Entra ID upstream-authentication filter that acquires an OAuth2
bearer token via the client-credentials grant, caches it, refreshes it
in the background before expiry, and injects "Authorization: Bearer
<token>" on every proxied request to Azure OpenAI.

The filter injects only the Authorization header; unlike SigV4, an Entra
token is not bound to Host, path, or body, so routing stays the
operator's responsibility and the filter has no ordering constraint. The
client secret is read from an environment variable, never from YAML.

Token acquisition runs on a dedicated background thread so building or
hot-reloading a pipeline never blocks on a network round-trip. The
filter fails closed with 503 until the first token is acquired and
whenever the cached token is missing or expired.

Scope is the client-secret credential only; managed identity, client
certificates, and OIDC federation slot into the same cache/refresh
machinery as follow-ups.

Signed-off-by: szedan <szedan@redhat.com>

* fix(azure): harden azure_ad token handling per review

Address code-review findings on the azure_ad filter:

- Validate authority_host and tenant_id reject URL-structural characters
  (scheme, path, query, '@' userinfo override) before they are
  interpolated into the token endpoint URL. This prevents a misconfigured
  or malicious value from redirecting the secret-bearing token POST to an
  unintended host. Validation allows both GUID and domain-style tenants.

- Cap the expiry skew at half the token TTL so a short-lived token is
  cached with real remaining validity instead of already-expired, which
  previously caused every request to fail closed with 503 forever while
  the refresher kept fetching.

- Back off exponentially (30s base, doubling, capped at 15m) on repeated
  token-acquisition failures instead of retrying at a flat 30s, so an
  unreachable endpoint does not become a steady poll.

Add unit tests for URL-component validation, skew capping, backoff
growth/cap, and the short-TTL caching regression.

Signed-off-by: szedan <szedan@redhat.com>

* feat(azure): gate azure_ad filter behind experimental build flag

Put the azure_ad (Entra ID) upstream auth filter behind a new
azure-ad-filter cargo feature, off by default and activating the
experimental marker, mirroring http-callout-filter. The filter module,
its registration, and the example integration test are cfg-gated; the
server and integration-test crates forward the feature. Registry tests
assert presence with the feature and absence without it.

This lets the filter merge and iterate while its background token
refresher moves onto a future core primitive (praxis-proxy/praxis#1043).

Signed-off-by: szedan <szedan@redhat.com>

---------

Signed-off-by: szedan <szedan@redhat.com>
…y replac…" (praxis-proxy#821)

This reverts commit 7d1aef9.

Signed-off-by: szedan <szedan@redhat.com>
…s-proxy#799)

* fix(openai): preserve OGX HTTP responses

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit 8738f3bd19717c74d463e4c9ab05c08d9fe7e76c)

* docs(openai): clarify redirect response handling

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit a5bab9cf11a5b6f315b410d9253e11446829c486)

* test(openai): cover non-success file responses

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit 7c4bab199b795907be5dfcf3e2d589eb27c85375)

* test(openai): cover file search callout statuses

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit fa96e6c7248807117ae8fe04d0df14171de58012)

* chore(openai): document response test complexity

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit 57426f8b33b037f0f04092ee732502731042fb5e)

* test(openai): cover example non-2xx file responses

Prove resolver 502 safety, transparent /v1/files 429, and closed file-search failure.

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit 2b0878d5284deb1616aee9285020cdd9600532ea)

* chore(openai): format file resolve test constants

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
(cherry picked from commit 646cf3f6e5bd29e37738109601e1c7ac6b507622)

* test(openai): restore file-search failure-mode asserts

Closed reject should keep searching because it returns before apply_batch.
Open continue still needs the reinference function_call_output. Drop the
stale JSON-decode rustdoc on ApiClientError.

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

---------

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
Co-authored-by: Aslak Knutsen <aslak@4fs.no>
…xis-proxy#818)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@bb05f3f...37fe631)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…raxis-proxy#819)

Bumps [github/codeql-action/analyze](https://github.com/github/codeql-action) from 4.37.7 to 4.37.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@ff2f1c6...db488dd)

---
updated-dependencies:
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.37.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…axis-proxy#820)

Bumps [github/codeql-action/init](https://github.com/github/codeql-action) from 4.37.7 to 4.37.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@ff2f1c6...db488dd)

---
updated-dependencies:
- dependency-name: github/codeql-action/init
  dependency-version: 4.37.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
)

Bumps rust from 1.97-alpine to 1.98-alpine.

---
updated-dependencies:
- dependency-name: rust
  dependency-version: 1.98-alpine
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

* feat(token-count): report prompt cache token breakdown

Providers that support prompt caching return how much of the input was
served from cache and how much was written to it, but token_count
collapsed that detail into a single input total. Callers that price
cached input differently from fresh input had no way to recover the
split.

Record the breakdown alongside the existing counts as two new metadata
keys, token.cache_read and token.cache_write, for both JSON and SSE
responses:

- OpenAI reports cached reads via prompt_tokens_details.cached_tokens
  and has no cache-write count, so writes are reported as zero.
- Anthropic reports both directions via cache_read_input_tokens and
  cache_creation_input_tokens, in JSON bodies and in the streaming
  message_start event.
- Google reports cached reads via cachedContentTokenCount and has no
  cache-write count.
- Bedrock Converse and the InvokeModel header path carry no cache
  signal, so neither key is recorded there. Claude served through
  InvokeModel still reports the full breakdown via the Anthropic
  response shape.

Both keys are a breakdown of token.input, not an addition to it:
every provider above already includes the cached portion in the input
total, so summing them would double-count. The example config states
this so downstream filters do not have to rediscover it.

Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>

* fix(token-count): distinguish an unreported cache count from zero

Cache counts were stored as plain integers defaulting to zero, so a
provider that says nothing about prompt caching produced the same
metadata as one that reports no cache hit. A downstream biller cannot
tell "this provider never told us" from "this request cached nothing",
and the two carry very different meaning when reconciling spend.

Model both cache fields as `Option<u64>` so absence stays absent:

- `TokenUsage::cache_read` and `cache_write` become `Option<u64>`, and
  `with_cache` takes options rather than integers.
- `set_cache_token_usage` emits each metadata key only when the
  corresponding count is present, and records a provider-reported `0`
  as "0" rather than dropping it.
- Streaming accumulators only insert a cache entry when the event
  carried one, so a later event without cache data no longer overwrites
  an earlier reported count with zero.
- `parse_anthropic_event` no longer fabricates `Some(0)` for a
  `message_delta`, which never carries cache fields at all.

The OpenAI and Google shapes stay inclusive (cached tokens are a subset
of the input count) while Anthropic stays exclusive (cache read and
write add to the input count); only the absent-versus-zero distinction
changes.

Part of praxis-proxy#577.

Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>

* fix(deps): update h2 to 0.4.16 for RUSTSEC-2026-0258

h2 0.4.15 is flagged by RUSTSEC-2026-0258 (unbounded empty DATA
frames), failing the dependency-check and security-audit jobs on
every branch. Bump the locked version to the patched 0.4.16 release.

Lockfile-only change: 0.4.16 is a drop-in patch with an unchanged
dependency list, so only the h2 version and checksum move.

Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>

* refactor(token_usage): extract record_json_usage from handle_json_body

Merging main into the prompt-cache branch recombined the overflow-status
guard (praxis-proxy#782) with the cache breakdown recording, pushing handle_json_body
one line over the too_many_lines threshold. Extract the end-of-stream
JSON usage recording into a dedicated helper to keep both concerns and
stay under the limit.

Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>

---------

Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>
Signed-off-by: Aslak Knutsen <aslak@4fs.no>
Co-authored-by: crstrn13 <acristur@redhat.com>
Co-authored-by: Aslak Knutsen <aslak@4fs.no>
Signed-off-by: Alexander Cristurean <acristur@redhat.com>
@leseb

leseb commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

@crstrn13 please rebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.