ci: prototype vllm-vcr backend for faster Responses integration tests - #719
Draft
leseb wants to merge 589 commits into
Draft
ci: prototype vllm-vcr backend for faster Responses integration tests#719leseb wants to merge 589 commits into
leseb wants to merge 589 commits into
Conversation
…raxis-proxy#721) Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
…xis-proxy#733) Signed-off-by: Sébastien Han <seb@redhat.com>
…-proxy#718) Signed-off-by: szedan <szedan@redhat.com>
…roxy#734) Signed-off-by: Sébastien Han <seb@redhat.com> Co-authored-by: Francisco Javier Arceo <farceo@redhat.com>
…xis-proxy#724) Signed-off-by: Sébastien Han <seb@redhat.com>
…xy#742) Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
…s-proxy#743) Signed-off-by: Sébastien Han <seb@redhat.com>
…axis-proxy#740) Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
…raxis-proxy#750) Signed-off-by: Sébastien Han <seb@redhat.com>
…xis-proxy#749) Signed-off-by: Sébastien Han <seb@redhat.com>
…axis-proxy#722) Signed-off-by: Sébastien Han <seb@redhat.com>
…SSE parser (praxis-proxy#739) Signed-off-by: Sébastien Han <seb@redhat.com>
…d Content-Type (praxis-proxy#745) Signed-off-by: Sébastien Han <seb@redhat.com>
To migrate to this repository we simply based on the core repository, and this patch removes, modifies and updates everything to match the new AI filters and capabilities crates we want here. This method allowed us to keep all git history and attribution. Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
Signed-off-by: Shane Utt <shaneutt@linux.com>
…xis-proxy#228) Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Signed-off-by: dependabot[bot] <support@github.com>
…s-proxy#223) Signed-off-by: dependabot[bot] <support@github.com>
…axis-proxy#225) Signed-off-by: Dimitri Saridakis <dimitri.saridakis@gmail.com> Signed-off-by: dimakis <dimitri.saridakis@gmail.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
… input arrays (praxis-proxy#619) * fix(openai_responses): make input_items pagination usable for ID-less input arrays Ordinary array `input` items (plain content-part objects) carry no `id` field, so `first_id`/`last_id` in the input_items list response stayed `null` even when `has_more` was true, leaving clients with no `after` value to resume pagination. Assign a stable synthetic id (`msg_{response_id}_input_{index}`) to any input item missing one, keyed to its original stored position so it stays stable regardless of sort order. IDs become the primary pagination cursor; the existing numeric offset fallback in `build_input_items_ok` now also covers the residual case where an array entry isn't a JSON object and can't carry a synthetic id. Signed-off-by: mkoushni <mkoushni@redhat.com> * test(openai_responses): cover mixed existing/synthetic id pagination Add a regression test for input arrays that mix items already carrying an id with id-less items, verifying ensure_stable_ids preserves the existing id and assigns a synthetic one only to the id-less item, and that cursor pagination (after=<existing>, then after=<synthetic>) resolves correctly across the mix. Addresses praxis-bot review feedback on praxis-proxy#612. Signed-off-by: mkoushni <mkoushni@redhat.com> --------- Signed-off-by: mkoushni <mkoushni@redhat.com>
…raxis-proxy#650) Bumps [github/codeql-action/analyze](https://github.com/github/codeql-action) from 4.37.3 to 4.37.4. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@e4fba86...f205ea1) --- updated-dependencies: - dependency-name: github/codeql-action/analyze dependency-version: 4.37.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…axis-proxy#652) Bumps [github/codeql-action/init](https://github.com/github/codeql-action) from 4.37.3 to 4.37.4. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@e4fba86...f205ea1) --- updated-dependencies: - dependency-name: github/codeql-action/init dependency-version: 4.37.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* implement and integrate is_promotable_value Signed-off-by: Harrison Stropkay <hstropka@redhat.com> * add oversized_model_not_promoted_to_header_or_results_or_metadata to model_rewrite/tests.rs Signed-off-by: Harrison Stropkay <hstropka@redhat.com> --------- Signed-off-by: Harrison Stropkay <hstropka@redhat.com> Co-authored-by: Francisco Javier Arceo <farceo@redhat.com>
…search (praxis-proxy#493) * feat(openai): add file_search_callout filter for hosted vector store search Add the `openai_file_search_callout` filter for executing OpenAI Responses API `file_search_call` output items against any vector store API compatible backend. The filter runs inside `iterative_request_router` so a model response can trigger vector store search and another inference within the same client request. vLLM backends that emit `function_call(name=file_search)` instead of native `file_search_call` are automatically translated before execution. Key capabilities: - Bounded fan-out (64 specs, 8-way concurrency) - Cross-store ranked aggregation with max_num_results - Citation tracking via ResponsesState.citation_files - SSRF-safe URL validation at startup - Fail-open (ignore) and fail-closed (reject) error policies - Partial failure tracking with incomplete status Authentication is delegated to upstream header injection or cluster-level request_headers_to_add rather than per-filter config. Closes praxis-proxy#29. Signed-off-by: Sébastien Han <seb@redhat.com> * fix(openai): preserve completed searches at deadline Record synthetic deadline failures only for specs at or after the scheduler cursor. Completed concurrency chunks have already produced and merged their results, so marking them failed could incorrectly reject the request or downgrade the call to incomplete. Also mark top-of-loop deadline handling as recorded to avoid duplicating failures after result sorting, and cover the scheduler boundary with a focused regression test. Signed-off-by: Sébastien Han <seb@redhat.com> * fix(file_search): inject shared SubRequestClient instead of private connector Replace the hardcoded SubRequestConnector::new(4, None) in file_search_callout with the shared server-level SubRequestClient, following the same pattern as file_resolve, compact, and web_search. Add from_config_with_client to FileSearchCalloutFilter and wire closure-based registration in register.rs so the filter inherits runtime.subrequest_pool_size and subrequest_max_connections from the server config. Signed-off-by: Sébastien Han <seb@redhat.com> * fix(file_search): add forward_headers and align failure vocabulary - Add `forward_headers` config field to forward request headers (e.g. Authorization) from the incoming request to the vector store callout, using the shared `ApiClient` header-forwarding mechanism already used by file_resolve. - Replace custom `on_error: ignore|reject` with `callout_failure_mode: open|closed` using the shared `FailureMode` enum from `config_validation`, matching the vocabulary used by `openai_responses_compact`. - Clarify `max_state_bytes` documentation: the filter and the enclosing iterative router may use different values because the smaller limit wins at runtime. - Update example config, integration test config, generated filter docs, and all unit tests. Signed-off-by: Sébastien Han <seb@redhat.com> --------- Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#688) extract_arguments used Value::to_string() which re-serializes JSON string values, wrapping them in an extra layer of quotes and escapes. Extract a shared normalize_arguments helper for the string-vs-non-string dispatch used by both the approval and execution paths. Closes praxis-proxy#558 Signed-off-by: Sébastien Han <seb@redhat.com>
* chore(deps): update Praxis crates to v0.5.2 Bump praxis-core, praxis-filter, praxis-protocol, praxis-tls, and praxis from v0.5.1 to v0.5.2. Since v0.5.2 is not yet published on crates.io, a temporary [patch.crates-io] section pins the crates to the v0.5.2 git tag. The new HttpFilterContext.metrics_route field (added in v0.5.2) is initialized in both test context constructors (apis, filters). Closes praxis-proxy#681 Signed-off-by: Sébastien Han <seb@redhat.com> * fix(ci): use inline git deps instead of patch section Replace the [patch.crates-io] section with inline git tag references in workspace dependencies. The patch approach conflicted with the test-praxis-main CI job (which appends its own patch section) and was rejected by cargo-deny (unknown-git = "deny"). Add https://github.com/praxis-proxy/praxis to deny.toml allow-git until v0.5.2 is published on crates.io. Signed-off-by: Sébastien Han <seb@redhat.com> --------- Signed-off-by: Sébastien Han <seb@redhat.com>
…criminator bodies (praxis-proxy#594) * fix(classifier): treat POST /v1/responses as authoritative for no-discriminator bodies A valid Responses create body that omits the discriminator fields the body heuristics key on (input, prompt object, previous_response_id, conversation) — e.g. `{"model":"gpt-5"}` — classified as unknown_json, which could be rejected or misrouted despite being a legitimate Responses create request. Treat the POST /v1/responses endpoint as authoritative: when body classification yields UnknownJson, promote it to Responses while keeping the extracted routing facts. Bodies with positive signals for another format (ChatCompletions / AnthropicMessages) and genuine parse failures (NonJson / InvalidJson) are left untouched so real errors still reject. Fixes praxis-proxy#413 Signed-off-by: Charlie Doern <cdoern@redhat.com> * test(classifier): update integration tests for authoritative create endpoint The unknown_json routing/reject integration tests posted a no-discriminator body to POST /v1/responses, which now classifies as Responses. Retarget them to a non-create path so they still exercise unknown_json handling, and add a test asserting a no-discriminator create body routes to the responses cluster. Signed-off-by: Charlie Doern <cdoern@redhat.com> * docs(classifier): document authoritative create endpoint and add reject-path test Address PR review feedback: - Convert inline comments in test bodies to assertion messages, per the project convention of putting explanations in the assertion's message argument so they print on failure. - Add `post_v1_responses_create_without_discriminator_not_rejected`, covering a valid no-discriminator create body under `on_invalid: reject` and closing the gap between the `continue`-mode and invalid-JSON tests. - Document the authoritative `POST /v1/responses` classification in the `ResponsesFormatFilter` doc comment, regenerate the filter doc, and describe the behavior in the format-routing example config. Signed-off-by: Charlie Doern <cdoern@redhat.com> * fix(responses): format merged classifier imports Signed-off-by: Charlie Doern <cdoern@redhat.com> --------- Signed-off-by: Charlie Doern <cdoern@redhat.com>
* fix(ci): stabilize flaky vLLM integration job Lower CPU memory reservation from 70% to 50% — the runner's ~15.6 GiB total minus other processes (cargo build, OGX, docker) often left less than the 10.93 GiB that 0.7 demanded, causing WorkerProc to crash on startup. The Qwen3-0.6B model needs ~1.2 GiB so 0.5 (~7.8 GiB) is plenty. Also add a container liveness check in the readiness loop so a crashed vLLM fails immediately instead of curling a dead container for 15 minutes. Signed-off-by: Sébastien Han <seb@redhat.com> * fix(ci): print full vLLM logs on container crash Signed-off-by: Sébastien Han <seb@redhat.com> --------- Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#717) Bumps [github/codeql-action/analyze](https://github.com/github/codeql-action) from 4.37.4 to 4.37.6. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@f205ea1...5595cca) --- updated-dependencies: - dependency-name: github/codeql-action/analyze dependency-version: 4.37.6 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…axis-proxy#718) Bumps [github/codeql-action/init](https://github.com/github/codeql-action) from 4.37.4 to 4.37.6. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@f205ea1...5595cca) --- updated-dependencies: - dependency-name: github/codeql-action/init dependency-version: 4.37.6 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(server): resolve external-filter discovery for optional dependencies
collect_root_deps previously matched packages by the literal name
"praxis-proxy" to find the AI server crate, colliding with the core
alias praxis = { package = "praxis-proxy" }. Anchor on cargo_metadata's
resolved root instead. Move the discovery logic into a new
praxis-ai-build-support crate so it's unit-testable, add an e2e test,
and fix container build + CI test-dependency issues found in review.
Closes praxis-proxy#478
Signed-off-by: mkoushni <mkoushni@redhat.com>
* fix(container): copy real build_support source instead of stubbing it
server/build.rs is a build-dependency consumer of praxis-ai-build-support:
cargo compiles build.rs (and its dependencies) up front, before any of the
crate's own source is available. Stubbing build_support/src/lib.rs to
`//! stub` therefore compiled a crate that exported none of the functions
build.rs calls (ActiveFeatures, discover_external_filter_crate_names,
etc.), breaking build.rs's own compilation and failing the entire
container build.
Copy the real build_support source alongside build.rs in the cache-build
stage instead, and drop it from both the stub-generation step and the
later cache-tricks re-copy, since it never needs the stub-then-replace
cycle used for the project's other crates.
Verified with a full `make container` build.
Signed-off-by: mkoushni <mkoushni@redhat.com>
* style(build_support): move test module to end of file, add section separators
Address praxis-bot review feedback on praxis-proxy#592: #[cfg(test)] mod tests must be
the last item in the file per convention, but it was declared right after
the imports. Move it after resolve_or_panic.
Also add the missing Public Types / Public Functions / Private Helpers
separator comments the file was otherwise missing, matching the file
ordering convention documented in CONTRIBUTING.md.
Signed-off-by: mkoushni <mkoushni@redhat.com>
* style: remove doc comment on test fn, add missing test-utilities separator
Address further praxis-bot review feedback on praxis-proxy#592:
- server/tests/external_filter_discovery_e2e.rs: drop the doc comment on
the #[test] fn; convention is that the function name is the
documentation, and the eprintln! already explains the skip-when-missing-
sibling-checkout behavior.
- server/build_support/src/tests.rs: add the missing `// Test Utilities`
separator before the fixture-building helpers, per the test-file
ordering convention in CONTRIBUTING.md.
Signed-off-by: mkoushni <mkoushni@redhat.com>
* fix(build_support): panic when root package has no matching resolve node
Addresses praxis-bot review: collect_root_deps previously fell back to
an empty vec via unwrap_or_default() if resolve.root had no matching
node in resolve.nodes, silently mirroring the old no-discovery
failure mode instead of panicking with an actionable message.
Signed-off-by: mkoushni <mkoushni@redhat.com>
---------
Signed-off-by: mkoushni <mkoushni@redhat.com>
…proxy#687) * feat(store): add validate_get_response_query_params Add a validation-only function that rejects unsupported (stream=true, include, starting_after, include_obfuscation), invalid, and unknown query parameters on GET /v1/responses/{id}. Keys and values are percent-decoded before validation; stream=false is accepted. Ref: praxis-proxy#555 Signed-off-by: Sébastien Han <seb@redhat.com> * feat(store): validate query params on GET /v1/responses/{id} Wire validate_get_response_query_params into handle_get_response before store initialization. Invalid, unsupported, or unknown query parameters now return 400 instead of being silently ignored. Closes: praxis-proxy#555 Signed-off-by: Sébastien Han <seb@redhat.com> * fix(store): strict percent-decoding, inline retrieval, new unit tests Use strict decode_utf8() with borrowed Cow values instead of decode_utf8_lossy/into_owned. Pass decoded values into validation so percent-encoded stream=false (e.g. stream=%66alse) is accepted. Inline retrieve_and_respond back into handle_get_response. Fix include integration test to use encoded key with valid value (include%5B%5D=reasoning.encrypted_content). Add unit tests for: percent-encoded stream=false accepted, invalid UTF-8 key rejected, invalid UTF-8 value rejected. Ref: praxis-proxy#555 Signed-off-by: Sébastien Han <seb@redhat.com> * style(store): fix nightly rustfmt attribute formatting Expand #[expect] attribute to multi-line format to match CI's nightly rustfmt output. Ref: praxis-proxy#555 Signed-off-by: Sébastien Han <seb@redhat.com> * test(store): add sync test for known params and validator match arms Addresses praxis-bot review feedback: ensures every entry in GET_RESPONSE_KNOWN_PARAMS is handled by validate_get_response_param without producing an "Unknown" error, preventing silent drift between the constant and the match arms. Signed-off-by: Sébastien Han <seb@redhat.com> * docs(store): document GET query parameter validation in architecture docs Operators reading the architecture docs will now learn that GET /v1/responses/{id} validates query parameters and rejects unsupported ones with a 400 response. Signed-off-by: Sébastien Han <seb@redhat.com> --------- Signed-off-by: Sébastien Han <seb@redhat.com>
Add a branch-scoped CI workflow that replaces the real vLLM CPU engine with vllm-vcr random tokens, eliminating the ~8 minute model startup penalty. The test file gains a VLLM_TEST_BACKEND=vcr-random mode that runs 3 protocol-safe tests (smoke, store/retrieve, streaming) while skipping 7 that require real inference or captured tool-call traces. Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#686) * fix(openai_responses): place web-search sources inside action object Move sources from the top-level output item into action.sources and encode each source as {"type":"url","url":"..."} to match the OpenAI Responses API web_search_call schema. Signed-off-by: Sébastien Han <seb@redhat.com> * fix(openai_responses): gate action.sources on include field Only emit action.sources in web_search_call output items when the request includes "web_search_call.action.sources", matching the include-controlled response contract. Signed-off-by: Sébastien Han <seb@redhat.com> * style(openai_responses): move INCLUDE_ACTION_SOURCES to constants section Group the include-gating constant with the other module constants at the top of the file, matching the existing separator convention. Signed-off-by: Sébastien Han <seb@redhat.com> --------- Signed-off-by: Sébastien Han <seb@redhat.com>
praxis-proxy#690) The openai_mcp_tool_resolve filter applied max_body_bytes only when buffering the original client body via StreamBuffer, but after expanding MCP tool definitions into function tools the rewritten body could exceed the limit unchecked. Add a post-serialization size guard that returns HTTP 413 before committing the expanded body. Signed-off-by: Sébastien Han <seb@redhat.com>
…-proxy#689) * fix: reject invalid query params in conversation item listing Make parse_item_list_params fallible. Reject malformed limit values, invalid order enums, unknown parameters, duplicates, and key-only params with 400 invalid_request_error instead of silently defaulting. Move param parsing before the conversation store lookup so malformed requests consistently return 400 rather than sometimes 404. Remove effective_limit clamping (now validated at parse time) and decode_query_component (replaced by strict decoding). Closes praxis-proxy#547 Signed-off-by: Sébastien Han <seb@redhat.com> * test: update and add unit tests for strict query validation Update existing tests to assert errors instead of silent defaults. Remove effective_limit and decode_query_component tests (dead code). Add tests for: limit=0, limit above max, duplicates, unknown params, key-only params, empty after, invalid UTF-8, empty components, and encoded duplicate keys. Signed-off-by: Sébastien Han <seb@redhat.com> * test: add integration tests for query validation error responses Verify handler-level 400 responses for invalid limit, invalid order, unknown params, and duplicate params. Also verify limit=0 returns an empty 200 page with has_more=false. Signed-off-by: Sébastien Han <seb@redhat.com> * test: add validation precedence regression test Verify that an invalid query on a nonexistent conversation returns 400, not 404, confirming that query validation runs before the store lookup. Signed-off-by: Sébastien Han <seb@redhat.com> * test: add boundary test for limit at MAX_PAGE_LIMIT Addresses praxis-bot review comment on PR praxis-proxy#689. Signed-off-by: Sébastien Han <seb@redhat.com> * style: fix rustfmt formatting in boundary test Signed-off-by: Sébastien Han <seb@redhat.com> --------- Signed-off-by: Sébastien Han <seb@redhat.com>
Increase all timeout values in the vLLM Responses SDK integration tests to 300 seconds to reduce flaky failures from slow model inference: - read_timeout_ms: 120000 → 300000 - timeout_ms: 120000 → 300000 - step_timeout_ms: 120000 → 300000 - SDK client timeout: 180 → 300 (all three client fixtures) Signed-off-by: Sébastien Han <seb@redhat.com>
…raxis-proxy#704) feat(openai): resolve connector_id through configured MCP connectors (praxis-proxy#316) Add connector_id resolution to the openai_mcp_tool_resolve filter, allowing requests to reference named MCP connectors instead of embedding raw server_url values. Configured connectors map short IDs to validated server URLs, keeping client payloads free of infrastructure details. Key changes: - ConnectorConfig with validation (max 64 connectors, 128-byte ID limit) - EntryResolution enum (PassThrough vs Resolved) preventing connector_id leak - resolve_connector_ids() with request-side ID length cap - URL redaction in client-facing connector errors - Zero-tool tool_choice rejection for resolved labels - Integration tests and example config Signed-off-by: Sébastien Han <seb@redhat.com>
franciscojavierarceo
approved these changes
Aug 14, 2026
franciscojavierarceo
left a comment
franciscojavierarceo
left a comment
Contributor
There was a problem hiding this comment.
the branch-scoped vllm-vcr prototype is appropriately isolated, keeps the existing live-inference coverage intact, and uses pinned actions, image digests, and a checksum for the experimental lane. the protocol-safe tests and explicit skips are consistent with the current testing patterns, and no actionable correctness or security issues stood out.
github-project-automation
Bot
moved this to Next
in AI Gateway - Provider APIs + Agentic Loop
praxis-bot
reviewed
Aug 21, 2026
praxis-bot
left a comment
praxis-bot
left a comment
Collaborator
There was a problem hiding this comment.
PR Review
Summary: Branch-scoped CI prototype adding a vllm-vcr-backed test lane and conditional skip markers for tests requiring real inference or captured tool-call traces.
Overall: Clean and well-structured. The workflow follows supply-chain best practices (SHA256-verified binary download, digest-pinned container image, commit-pinned actions, minimal contents: read permissions). Readiness loops have proper timeout handling with early-exit on process/container death. Test changes correctly gate VCR-mode assertions without weakening live-mode coverage -- test_store_and_retrieve's retrieval assertion change to response.status is safe because the live-mode branch already hard-asserts completed above it. No Critical, Large, or Medium findings.
| Severity | Count |
|---|---|
| Critical | 0 |
| Large | 0 |
| Medium | 0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prototype a vllm-vcr-backed CI lane to measure the speedup from
eliminating the ~8 minute vLLM CPU model startup penalty. The
branch-scoped workflow runs the vLLM frontend with vllm-vcr random
tokens instead of real inference. The test file gains a
VLLM_TEST_BACKEND=vcr-randommode that runs 3 protocol-safe tests(smoke, store/retrieve, streaming) and skips 7 that need real model
inference or captured tool-call traces.
Related issue
N/A — exploratory prototype to evaluate CI runtime savings.
Validation
make lintmake buildChecklist
Signed-off-bytrailer.Breaking changes
None — branch-scoped prototype workflow, no changes to existing CI or test behavior.