Skip to content

feat(inference): add llmisvc_model_provider_resolver - #699

Open
jland-redhat wants to merge 600 commits into
praxis-proxy:mainfrom
jland-redhat:llmisvc_model_provider_resolver
Open

feat(inference): add llmisvc_model_provider_resolver#699
jland-redhat wants to merge 600 commits into
praxis-proxy:mainfrom
jland-redhat:llmisvc_model_provider_resolver

Conversation

@jland-redhat

@jland-redhat jland-redhat commented Aug 10, 2026

Copy link
Copy Markdown

Summary

  • Add llmisvc_model_provider_resolver, porting only the LLMISvc / KServe BBR body-rewrite path from IPP’s model-provider-resolver.
  • Prefer a configurable model header (default X-Model, aligned with model_to_header), fall back to body "model", and when the value is a publisher ID (publishers/.../models/<name>) rewrite the body "model" to <name> only.
  • Leave the routing header untouched so KServe can still route on the publisher ID; stash the original ID in llmisvc_model_provider_resolver.publisher_id for metering.
  • Includes unit tests, example config, integration coverage, and generated filter docs.

Does not port ExternalModel / ExternalProvider resolution, weighted provider selection, Host rewrite, api-format detection, or credential handling.

Sister PR (merge after this)

Without the ExtProc follow-up, body rewrites that change length will fail in Envoy BUFFERED + header SEND mode even though this filter’s rewrite is correct.

Test plan

  • Unit tests for rewrite / header preference / body fallback / non-publisher passthrough
  • Example config + integration tests
  • Validated on local cluster with publisher-ID model request; upstream returned a completion:
{
  "id": "chatcmpl-efb19481-6952-5be9-9572-ebfb8aa9070b",
  "model": "demo/sim-stream",
  "object": "chat.completion",
  "choices": [
    {
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "I am fine, how are you today? ..."
      }
    }
  ]
}

franciscojavierarceo and others added 30 commits June 30, 2026 13:39
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
…SSE parser (praxis-proxy#739)

Signed-off-by: Sébastien Han <seb@redhat.com>
…d Content-Type (praxis-proxy#745)

Signed-off-by: Sébastien Han <seb@redhat.com>
To migrate to this repository we simply based on the core repository,
and this patch removes, modifies and updates everything to match the
new AI filters and capabilities crates we want here. This method allowed
us to keep all git history and attribution.

Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Signed-off-by: dependabot[bot] <support@github.com>
…axis-proxy#225)

Signed-off-by: Dimitri Saridakis <dimitri.saridakis@gmail.com>
Signed-off-by: dimakis <dimitri.saridakis@gmail.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
dependabot Bot and others added 8 commits August 12, 2026 08:16
…axis-proxy#718)

Bumps [github/codeql-action/init](https://github.com/github/codeql-action) from 4.37.4 to 4.37.6.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@f205ea1...5595cca)

---
updated-dependencies:
- dependency-name: github/codeql-action/init
  dependency-version: 4.37.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(server): resolve external-filter discovery for optional dependencies

collect_root_deps previously matched packages by the literal name
"praxis-proxy" to find the AI server crate, colliding with the core
alias praxis = { package = "praxis-proxy" }. Anchor on cargo_metadata's
resolved root instead. Move the discovery logic into a new
praxis-ai-build-support crate so it's unit-testable, add an e2e test,
and fix container build + CI test-dependency issues found in review.

Closes praxis-proxy#478

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(container): copy real build_support source instead of stubbing it

server/build.rs is a build-dependency consumer of praxis-ai-build-support:
cargo compiles build.rs (and its dependencies) up front, before any of the
crate's own source is available. Stubbing build_support/src/lib.rs to
`//! stub` therefore compiled a crate that exported none of the functions
build.rs calls (ActiveFeatures, discover_external_filter_crate_names,
etc.), breaking build.rs's own compilation and failing the entire
container build.

Copy the real build_support source alongside build.rs in the cache-build
stage instead, and drop it from both the stub-generation step and the
later cache-tricks re-copy, since it never needs the stub-then-replace
cycle used for the project's other crates.

Verified with a full `make container` build.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* style(build_support): move test module to end of file, add section separators

Address praxis-bot review feedback on praxis-proxy#592: #[cfg(test)] mod tests must be
the last item in the file per convention, but it was declared right after
the imports. Move it after resolve_or_panic.

Also add the missing Public Types / Public Functions / Private Helpers
separator comments the file was otherwise missing, matching the file
ordering convention documented in CONTRIBUTING.md.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* style: remove doc comment on test fn, add missing test-utilities separator

Address further praxis-bot review feedback on praxis-proxy#592:

- server/tests/external_filter_discovery_e2e.rs: drop the doc comment on
  the #[test] fn; convention is that the function name is the
  documentation, and the eprintln! already explains the skip-when-missing-
  sibling-checkout behavior.
- server/build_support/src/tests.rs: add the missing `// Test Utilities`
  separator before the fixture-building helpers, per the test-file
  ordering convention in CONTRIBUTING.md.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(build_support): panic when root package has no matching resolve node

Addresses praxis-bot review: collect_root_deps previously fell back to
an empty vec via unwrap_or_default() if resolve.root had no matching
node in resolve.nodes, silently mirroring the old no-discovery
failure mode instead of panicking with an actionable message.

Signed-off-by: mkoushni <mkoushni@redhat.com>

---------

Signed-off-by: mkoushni <mkoushni@redhat.com>
…proxy#687)

* feat(store): add validate_get_response_query_params

Add a validation-only function that rejects unsupported (stream=true,
include, starting_after, include_obfuscation), invalid, and unknown
query parameters on GET /v1/responses/{id}. Keys and values are
percent-decoded before validation; stream=false is accepted.

Ref: praxis-proxy#555
Signed-off-by: Sébastien Han <seb@redhat.com>

* feat(store): validate query params on GET /v1/responses/{id}

Wire validate_get_response_query_params into handle_get_response
before store initialization. Invalid, unsupported, or unknown query
parameters now return 400 instead of being silently ignored.

Closes: praxis-proxy#555
Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(store): strict percent-decoding, inline retrieval, new unit tests

Use strict decode_utf8() with borrowed Cow values instead of
decode_utf8_lossy/into_owned. Pass decoded values into validation
so percent-encoded stream=false (e.g. stream=%66alse) is accepted.

Inline retrieve_and_respond back into handle_get_response. Fix
include integration test to use encoded key with valid value
(include%5B%5D=reasoning.encrypted_content).

Add unit tests for: percent-encoded stream=false accepted, invalid
UTF-8 key rejected, invalid UTF-8 value rejected.

Ref: praxis-proxy#555
Signed-off-by: Sébastien Han <seb@redhat.com>

* style(store): fix nightly rustfmt attribute formatting

Expand #[expect] attribute to multi-line format to match CI's
nightly rustfmt output.

Ref: praxis-proxy#555
Signed-off-by: Sébastien Han <seb@redhat.com>

* test(store): add sync test for known params and validator match arms

Addresses praxis-bot review feedback: ensures every entry in
GET_RESPONSE_KNOWN_PARAMS is handled by validate_get_response_param
without producing an "Unknown" error, preventing silent drift between
the constant and the match arms.

Signed-off-by: Sébastien Han <seb@redhat.com>

* docs(store): document GET query parameter validation in architecture docs

Operators reading the architecture docs will now learn that
GET /v1/responses/{id} validates query parameters and rejects
unsupported ones with a 400 response.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#686)

* fix(openai_responses): place web-search sources inside action object

Move sources from the top-level output item into action.sources and
encode each source as {"type":"url","url":"..."} to match the OpenAI
Responses API web_search_call schema.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(openai_responses): gate action.sources on include field

Only emit action.sources in web_search_call output items when the
request includes "web_search_call.action.sources", matching the
include-controlled response contract.

Signed-off-by: Sébastien Han <seb@redhat.com>

* style(openai_responses): move INCLUDE_ACTION_SOURCES to constants section

Group the include-gating constant with the other module constants
at the top of the file, matching the existing separator convention.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
praxis-proxy#690)

The openai_mcp_tool_resolve filter applied max_body_bytes only when
buffering the original client body via StreamBuffer, but after expanding
MCP tool definitions into function tools the rewritten body could exceed
the limit unchecked. Add a post-serialization size guard that returns
HTTP 413 before committing the expanded body.

Signed-off-by: Sébastien Han <seb@redhat.com>
…-proxy#689)

* fix: reject invalid query params in conversation item listing

Make parse_item_list_params fallible. Reject malformed limit values,
invalid order enums, unknown parameters, duplicates, and key-only
params with 400 invalid_request_error instead of silently defaulting.

Move param parsing before the conversation store lookup so malformed
requests consistently return 400 rather than sometimes 404.

Remove effective_limit clamping (now validated at parse time) and
decode_query_component (replaced by strict decoding).

Closes praxis-proxy#547

Signed-off-by: Sébastien Han <seb@redhat.com>

* test: update and add unit tests for strict query validation

Update existing tests to assert errors instead of silent defaults.
Remove effective_limit and decode_query_component tests (dead code).
Add tests for: limit=0, limit above max, duplicates, unknown params,
key-only params, empty after, invalid UTF-8, empty components, and
encoded duplicate keys.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test: add integration tests for query validation error responses

Verify handler-level 400 responses for invalid limit, invalid order,
unknown params, and duplicate params. Also verify limit=0 returns an
empty 200 page with has_more=false.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test: add validation precedence regression test

Verify that an invalid query on a nonexistent conversation returns 400,
not 404, confirming that query validation runs before the store lookup.

Signed-off-by: Sébastien Han <seb@redhat.com>

* test: add boundary test for limit at MAX_PAGE_LIMIT

Addresses praxis-bot review comment on PR praxis-proxy#689.

Signed-off-by: Sébastien Han <seb@redhat.com>

* style: fix rustfmt formatting in boundary test

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
Increase all timeout values in the vLLM Responses SDK integration tests
to 300 seconds to reduce flaky failures from slow model inference:

- read_timeout_ms: 120000 → 300000
- timeout_ms: 120000 → 300000
- step_timeout_ms: 120000 → 300000
- SDK client timeout: 180 → 300 (all three client fixtures)

Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#704)

feat(openai): resolve connector_id through configured MCP connectors (praxis-proxy#316)

Add connector_id resolution to the openai_mcp_tool_resolve filter,
allowing requests to reference named MCP connectors instead of embedding
raw server_url values. Configured connectors map short IDs to validated
server URLs, keeping client payloads free of infrastructure details.

Key changes:
- ConnectorConfig with validation (max 64 connectors, 128-byte ID limit)
- EntryResolution enum (PassThrough vs Resolved) preventing connector_id leak
- resolve_connector_ids() with request-side ID length cap
- URL redaction in client-facing connector errors
- Zero-tool tool_choice rejection for resolved labels
- Integration tests and example config

Signed-off-by: Sébastien Han <seb@redhat.com>

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praxis-bot review (round 2)

Fix commit is clean: backtick formatting, CI metrics_route field, and test loop refactor all look correct. One nit below. The prior Medium (body mutation when "model" absent) remains open.

Findings: 1 nit

assert_eq!(
llmisvc_short_model_name("publishers/ns/models/a/b"),
Some("a/b"),
"SplitN keeps remainder after first /models/"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] Assertion message says SplitN but the implementation (line 279) uses split_once. They are semantically similar, but the message should match the actual method for accuracy.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — assertion message now says split_once.

* feat(openai): support finite chat continuations

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* fix(openai): fail closed for unresolved stream history

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* fix(test): add assertion messages and Responses live recording coverage

Add diagnostic messages to all bare assert! calls introduced in the
chat continuation feature to comply with repository conventions.

Extend live recording tests with a two-turn Responses scenario that
exercises the ${PREVIOUS_RESPONSE_ID} placeholder binding during
record_live, verifying the first response ID reaches the second
request.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Co-authored-by: Sébastien Han <seb@redhat.com>

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praxis-bot review (round 3)

Clean implementation with good separation of concerns. Config validation (deny_unknown_fields, empty-header rejection, validate_max_body_bytes), the header-vs-body resolution chain, and the split_once-based publisher-ID parsing are all correct. Integration tests exercise the core rewrite, routing-header preservation, and non-publisher passthrough end-to-end. The register.rs test refactor to a loop is a nice cleanup.

The prior Medium from round 1 (body mutation when "model" absent -- obj.insert(...) adds a field the caller never sent) remains the only actionable concern and is not yet addressed.

Findings: 0 new (prior Medium still open)

skamenan7 and others added 14 commits August 13, 2026 13:26
* Limit MCP dynamic values and add boundary tests

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

* fix(mcp): bound method and name filter results

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

---------

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
…xy#711)

* fix(responses): disable parallel tools on first inference

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

* test(inference): add fixture coverage for agentic parallel_tool_calls

Add a controlled synthetic scenario and recording that proves the
agentic loop injects `parallel_tool_calls: false` into the upstream
request when the client omits it. This closes the inference fixture
coverage gap for the dirty-marker implementation.

- Add `agentic-loop-fixture.yaml` example config (replay-safe subset
  of the agentic loop pipeline without external callout filters)
- Add `agentic_loop` and `iterative_request_router` to the replay
  filter allowlist (they make no external callouts)
- Add scenario, recording, and coverage.yaml entry for
  `responses.agentic.parallel_tool_calls`
- Update the snapshot test in `coverage.rs` for the new scope,
  feature, scenario, and recording counts
- Regenerate example and inference READMEs

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Co-authored-by: Sébastien Han <seb@redhat.com>
* feat(filters): add OpenTelemetry routing spans

Add feature-gated routing.select spans for successful intelligent_route decisions while leaving request lifecycle, propagation, sampling, and export ownership in Praxis core.

Keep the default build unchanged and avoid OpenTelemetry SDK dependencies in the AI filters. Record only validated, bounded routing attributes and document the ownership and privacy boundaries.

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>

* test(filters): align telemetry test conventions

Document the private routing-selection fields used to project bounded
OpenTelemetry attributes.

Move the candidate fixture below the tests, use the standard
test-utilities separator, and add diagnostic messages to every
assertion.

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>

---------

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
…-proxy#625)

* fix(conversations): align create and update request contracts

Align the locally owned Conversation create and update request bodies with confirmed OpenAI behavior. Create requests accept an absent body and nullable optional fields, while update requests require a non-null metadata object.

Generate the matching OpenAPI schemas from the shared runtime contracts, retain the 20-item limit, cover absent, null, empty, valid, and invalid body shapes, and document the verified live behavior where it contradicts the pinned upstream specification.

Closes praxis-proxy#566

Signed-off-by: Sébastien Han <seb@redhat.com>

* docs(conformance): remove unnecessary probe cleanup detail

Signed-off-by: Sébastien Han <seb@redhat.com>

* test(conversations): expect 400 for update without metadata

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(conversations): return OpenAI error codes on update validation

Map update-conversation parse and validation errors to the OpenAI error
contract with code (missing_required_parameter, invalid_type) and param
fields. Add inner anyOf assertions for the two-layer nullable metadata
schema and an array-type rejection test.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(conversations): narrow error code/param scope in update responses

- Remove hardcoded "but got null" from classify_update_error message
  since the error applies to all non-object types, not just null
- Restore invalid_input_response to not include code/param fields,
  keeping those only for update-specific errors via
  invalid_input_response_with
- Use invalid_type code only for actual type violations from
  validate_metadata, not for constraint violations (key/value length,
  key count)

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(conversations): address praxis-bot review findings

- Replace string-matching heuristic (msg.contains("must be")) with
  MetadataError enum that structurally distinguishes type errors from
  constraint violations
- Add maxItems assertion to OpenAPI schema test ensuring the 20-item
  bound is preserved in the generated document
- Handle empty update body with missing_required_parameter error code
  matching the {} case, as documented in conformance README

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
…proxy#725)

v0.5.2 is now published on crates.io, so remove the temporary git tag
workaround introduced in praxis-proxy#701.

Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#733)

* fix(a2a): defer route commit until EOS to prevent prefix-poisoning

GuardResult::Redact was returning FilterAction::Continue while recording
status=redacted (guardrails fix). Separately, the A2A non-streaming
capture path committed task-route ownership as soon as the JSON balance
scanner reported a structurally complete value, regardless of
end_of_stream. A backend could deliver a valid JSON task response with
end_of_stream=false, have the route stored, then append trailing garbage
to produce an overall-invalid response — poisoning task ownership while
the complete body was not valid JSON.

When the scanner reports is_complete but end_of_stream is false,
parse_to_tentative() now stores the parsed value in filter metadata under
a2a.response.tentative_json without committing to the route store.
Subsequent chunks are inspected:
- non-whitespace bytes → tentative discarded (trailing content proves
  the response is not valid JSON)
- end_of_stream with no further non-whitespace → commit_tentative_capture()
  promotes the held value into the store

If end_of_stream arrives together with is_complete, try_capture_from_buffer()
is called directly as before, preserving the common fast path.

Update tests:
- json_response_split_across_chunks_defers_capture_until_eos
- many_single_byte_chunks_capture_route_at_eos
- split_json_response_with_context_stores_context_route
- assert_capture_scratch_cleared includes a2a.response.tentative_json
Add regression tests:
- complete_json_prefix_then_garbage_does_not_capture_route
- complete_json_followed_by_whitespace_at_eos_captures_route

Fixes: fnd_sig-feat-custom-ai-agentic-class_e1322f3e62

Signed-off-by: mkoushni <mkoushni@redhat.com>

* test(a2a): add tentative-survives-whitespace-chunk regression test

Covers the keep-waiting branch of the tentative guard: a whitespace-only
chunk with end_of_stream=false must leave the tentative JSON intact without
committing the route; only the subsequent EOS callback should commit.

Reported by praxis-bot review of fix/a2a-defer-route-commit-until-eos.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* docs(a2a): correct try_capture_from_buffer doc comment

The previous comment claimed the function was called after a tentative parse
was promoted by a whitespace-only EOS callback; that path actually calls
commit_tentative_capture. Update to accurately describe the two real call
sites: the fast path (is_complete && end_of_stream simultaneously) and the
fallback where EOS arrives with an incomplete buffer.

Reported by praxis-bot review of fix/a2a-defer-route-commit-until-eos.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(a2a): resolve clippy too-many-lines and needless-pass-by-ref-mut

Extract the tentative-guard logic from handle_non_streaming_capture into a
dedicated handle_pending_tentative helper. This reduces handle_non_streaming_capture
to under the 30-line clippy limit while improving readability.

Drop the unused &mut on ctx in commit_tentative_capture — the function only
reads filter_metadata, so &HttpFilterContext<'_> is sufficient.

Remove two inline comments from json_response_split_across_chunks_defers_
capture_until_eos that violated the project convention (no inline comments
in test bodies) and pushed the function over the 30-line limit.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(a2a): clear capture state on serde_json serialization failure in parse_to_tentative

If serde_json::to_string fails inside parse_to_tentative, the tentative_json
key is not set but buffer_hex, balance state, and capture_enabled all remain.
Subsequent chunks re-enter the main flow and when EOS arrives is_complete ||
end_of_stream fires, committing the route via try_capture_from_buffer and
bypassing the tentative guard entirely.

In practice this is effectively infallible for a Value that was just
deserialized, but add a defense-in-depth else branch that calls
clear_capture_metadata so the tentative guard cannot be bypassed under
any failure mode.

Reported by praxis-bot review of fix/a2a-defer-route-commit-until-eos.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(a2a): extract assert_tentative_pending helper to fix too-many-lines lint

json_response_split_across_chunks_defers_capture_until_eos was 32 non-blank
lines (limit 30). Extract the paired tentative-state assertions into a shared
assert_tentative_pending helper, bringing the test well under the limit.

Also restore the #[expect(clippy::too_many_lines)] attribute to its correct
position on assert_capture_scratch_cleared after it was accidentally
displaced during the helper insertion.

Signed-off-by: mkoushni <mkoushni@redhat.com>

* fix(a2a): use RFC 8259 whitespace predicate and remove test inline comments

Use a JSON-specific whitespace predicate (`b' ' | b'\t' | b'\n' | b'\r'`)
instead of `is_ascii_whitespace()` which also matches vertical tab (\x0B)
and form feed (\x0C). RFC 8259 defines only four insignificant-whitespace
code points; using the broader ASCII predicate could allow non-JSON
trailing bytes to be silently treated as whitespace, bypassing the
tentative-guard discard branch.

Move the Clawpatch finding ID for `complete_json_prefix_then_garbage_
does_not_capture_route` into the first assertion message where it is
visible as traceability. Remove all other inline comments from new and
modified test function bodies per project conventions.

Signed-off-by: mkoushni <mkoushni@redhat.com>

---------

Signed-off-by: mkoushni <mkoushni@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
…lout (praxis-proxy#737)

feat(file_search): add full-flow-agentic example with IRR file search callout

Add a full-flow-agentic.yaml example config that wraps the inference
step in an iterative_request_router, enabling server-side file search
execution through vector store callouts. On IRR continuation iterations
the synthetic request lacks client credentials; callout_request_headers
now returns the original headers (via Cow to avoid cloning on iteration
0) with Connection-nominated hop-by-hop headers filtered out.

Signed-off-by: Sébastien Han <seb@redhat.com>
…-proxy#748)

* refactor(filters): rename agentic_loop filter to openai_agentic_loop

Align the agentic loop filter name with the existing OpenAI
Responses API filter naming convention (openai_mcp_dispatch,
openai_web_search, openai_tool_parse, etc.).

Updated across registration, filter impl, config validation,
YAML examples, unit/integration tests, docs, and fixture replay.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix: rename agentic_loop in vllm SDK integration test

Missed reference in test_openai_responses_vllm.py caught during
PR review.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
…y#739)

* ci(store): add PostgreSQL CI coverage for response store

Add CI workflows that exercise the response store with a real
PostgreSQL backend, covering both unit tests and end-to-end vLLM
integration tests.

New postgres.yaml workflow runs the 27 ignored store unit tests
(with a service container providing DATABASE_URL) and the 2 ignored
Rust integration tests (which spawn their own container).

The vLLM integration workflow gains a vllm-responses-postgres job
that reruns the full Python SDK test suite with STORE_BACKEND=postgres,
exercising persistence, rehydration, and conversations through the
PostgreSQL path.

Also fixes two pre-existing bugs in the ignored Postgres test
infrastructure: table name suffixes containing uppercase ThreadId
characters caused schema validation failures (PostgreSQL folds
unquoted identifiers to lowercase), and the Rust integration test
omitted ssl_mode: disable, causing VerifyFull handshake failures
against the plaintext test container.

Signed-off-by: Sébastien Han <seb@redhat.com>

* fix(ci): detect store backend from DATABASE_URL prefix

Drop the STORE_BACKEND env var and infer the backend type from the
DATABASE_URL scheme instead, addressing review feedback.

Signed-off-by: Sébastien Han <seb@redhat.com>

---------

Signed-off-by: Sébastien Han <seb@redhat.com>
…t Router (praxis-proxy#730)

* Add conformance test: inference fallback with protocol translation

First conformance test for the AI gateway. Validates that an
iterative_request_router composes correctly with
responses_to_chat_completions protocol translation and
credential_injection across a failover boundary.

Two tests exercise the example config:
- fallback_on_primary_503: primary returns 503, fallback
  receives the translated Chat Completions request with
  isolated credentials, client gets a Responses API resource.
- primary_succeeds_no_fallback: primary returns 200, fallback
  receives no requests.

Each IRR step re-runs openai_responses_format and
openai_responses_validate because the IRR resets per-step
metadata for credential isolation while preserving extensions.

Assisted by Opus 4.6

Signed-off-by: usize <mofoster@redhat.com>

* Document step boundary rules in config comments

Move the metadata/extension persistence explanation into the
example config header comments, where someone building a
pipeline encounters it. Add a one-liner to ai-inference.md
pointing to the example. Document real-world adaptation
(model rewrite, TLS/SNI, env_var credentials, Host header)
in the "Adapting for real providers" section.

Assisted by Opus 4.6

Signed-off-by: usize <mofoster@redhat.com>

* Address review feedback and sync with upstream

- Regenerate examples/README.md for new inference/ category
- Remove inline section comments from test bodies
- Add assertion messages to all bare assert_eq! calls
- Add both_backends_fail_returns_last_error test case
- Update Cargo.lock after rebase (git tags -> crates.io)

Assisted by Opus 4.6

Signed-off-by: usize <mofoster@redhat.com>

---------

Signed-off-by: usize <mofoster@redhat.com>
Resolve metrics_route test-utils conflict (now unconditional on
0.5.2) and keep both inference_fallback and llmisvc example mods.

Signed-off-by: jland <jland@redhat.com>
Assume model_to_header already set the routing header;
no-op when it is missing. Do not invent a body model
field, and rustfmt the llmisvc tests.

Signed-off-by: jland <jland@redhat.com>
Signed-off-by: jland <jland@redhat.com>
@Jaland
Jaland force-pushed the llmisvc_model_provider_resolver branch from a1d79f9 to b6c729b Compare August 14, 2026 16:50

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Re-Review

Summary: Previous findings (Medium: body mutation when model absent, Nit: SplitN assertion message) are both resolved. Commit 8656cdd5 added the guard to prevent inventing a body "model" field, and the test leaves_body_unchanged_when_header_publisher_id_but_no_body_model exercises this path. The assertion message now correctly references split_once. One new medium finding on missing test coverage.

Severity Count
Medium 1

// -----------------------------------------------------------------------------

/// Deserialized YAML config for the `LLMISvc` model-provider resolver.
#[derive(Debug, Deserialize)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] The config struct has #[serde(deny_unknown_fields)] but no test exercises this validation. While the annotation is present, untested validation paths can silently break if refactored.

Add a test:

#[test]
fn from_config_rejects_unknown_fields() {
    let yaml: serde_yaml::Value = serde_yaml::from_str(
        r#"
header: "X-Model"
bogus_field: true
"#,
    )
    .unwrap();
    assert!(
        LlmisvcModelProviderResolverFilter::from_config(&yaml).is_err(),
        "unknown fields should be rejected"
    );
}

Other filters in the codebase (e.g. identity_header_guard in PR #709) include this test to ensure deny_unknown_fields actually rejects invalid config.

@shaneutt
shaneutt self-requested a review as a code owner August 28, 2026 17:11
shaneutt pushed a commit that referenced this pull request Aug 28, 2026
Signed-off-by: Sébastien Han <seb@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.