feat(openai): stream Responses across IRR agentic rounds - #756
Conversation
3e8665c to
e060573
Compare
praxis-bot
left a comment
There was a problem hiding this comment.
PR Review
Summary: Adds terminal and multi-round agentic Responses SSE streaming through IRR, transitioning several filters from StreamBuffer to Stream body mode, adding terminal_streaming config to openai_responses_proxy, and implementing logical stream normalization in openai_stream_events.
Overall: Well-structured change with thorough test coverage across unit, integration, and inference fixture levels. The streaming dispatch guards (streamed_round_is_dispatchable, incomplete/failed stream protection) are sound. One control-flow issue in the error path.
| Severity | Count |
|---|---|
| Critical | 0 |
| Large | 0 |
| Medium | 1 |
| "openai_agentic_loop supports exactly one function call per round", | ||
| )?; | ||
| } | ||
| Ok(true) |
There was a problem hiding this comment.
[Medium] prepare_streamed_round returns Ok(true) after end_stream_with_error, which already sets ACTION_DONE and clears both call vectors. Returning true causes evaluate_loop_decision to run, re-enter the empty-tool-calls branch, call finalize_response_body (creating body bytes from response_object), and redundantly set ACTION_DONE again.
With logical_stream: true the body is overwritten by finalize_logical_stream, masking the issue. In a non-logical-stream configuration the serialized JSON leaks to the client after the SSE events.
Return Ok(false) after end_stream_with_error so the caller uses the early-return path (insert state, return Continue) instead of falling through to evaluate_loop_decision:
if state.tool_calls.len() > 1 {
end_stream_with_error(
ctx,
state,
"invalid_request_error",
"openai_agentic_loop supports exactly one function call per round",
)?;
return Ok(false);
}e060573 to
ce2302f
Compare
ce2302f to
7e6a463
Compare
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
The irr-terminal-streaming.yaml load_balancer cluster proxies to a loopback endpoint (127.0.0.1:3001). Praxis main hardened cluster endpoint validation to reject sensitive addresses unless insecure_options.allow_private_endpoints is set, which broke the all_example_configs_parse schema test under the praxis-main job. Add the top-level insecure_options.allow_private_endpoints override, matching every other example config that fronts local backends. Signed-off-by: Sébastien Han <seb@redhat.com>
7e6a463 to
425f41e
Compare
franciscojavierarceo
left a comment
There was a problem hiding this comment.
the checked-in agentic-loop example currently fails to parse because iterative_request_router rejects max_stream_response_bytes. this prevents the example from starting and causes the new integration, vLLM, and coverage jobs to fail before they exercise streaming. we need to remove or rename this setting to the router’s supported field, then rerun the new streaming coverage.
Summary
Adds terminal and multi-round agentic Responses SSE streaming through IRR using praxis-proxy/praxis#988 at
a198962d, preserving one logical response across MCP and web-search rounds.Includes bounded lifecycle and error handling, persistence integration, example configs, generated docs, synthetic inference fixtures, and live vLLM terminal-streaming coverage.
Related issue
Closes #706
Validation
cargo test -p praxis-ai-apis openai::responses::agentic_loop::testsandcargo test -p praxis-ai-apis openai::responses::stream_events::testscargo test -p praxis-tests-integration --test suite examples::openai_agentic_loop -- --nocaptureandcargo test -p praxis-tests-integration --test suite examples::irr_terminal_streaming -- --nocapturemake test-inference-fixturesuv run tests/integration/sdk/openai/test_openai_responses_vllm.py -s -vv -k test_streaming_through_irrmake buildmake lintChecklist
Signed-off-bytrailer.Breaking changes
None.