Skip to content

feat(openai): stream Responses across IRR agentic rounds - #756

Draft
leseb wants to merge 4 commits into
praxis-proxy:mainfrom
leseb:leseb/issue-706-implementation
Draft

feat(openai): stream Responses across IRR agentic rounds#756
leseb wants to merge 4 commits into
praxis-proxy:mainfrom
leseb:leseb/issue-706-implementation

Conversation

@leseb

@leseb leseb commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds terminal and multi-round agentic Responses SSE streaming through IRR using praxis-proxy/praxis#988 at a198962d, preserving one logical response across MCP and web-search rounds.
Includes bounded lifecycle and error handling, persistence integration, example configs, generated docs, synthetic inference fixtures, and live vLLM terminal-streaming coverage.

Related issue

Closes #706

Validation

  • Unit tests: cargo test -p praxis-ai-apis openai::responses::agentic_loop::tests and cargo test -p praxis-ai-apis openai::responses::stream_events::tests
  • Integration tests: cargo test -p praxis-tests-integration --test suite examples::openai_agentic_loop -- --nocapture and cargo test -p praxis-tests-integration --test suite examples::irr_terminal_streaming -- --nocapture
  • Inference fixtures: make test-inference-fixtures
  • Live backend: uv run tests/integration/sdk/openai/test_openai_responses_vllm.py -s -vv -k test_streaming_through_irr
  • make build
  • make lint

Checklist

  • I reviewed every changed line and can explain the change.
  • New capabilities include an example config and functional example test.
  • User-facing behavior and generated documentation are updated.
  • Performance-sensitive changes include appropriate benchmark or load-test evidence.
  • Commits are signed and include a Signed-off-by trailer.

Breaking changes

None.

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review

Summary: Adds terminal and multi-round agentic Responses SSE streaming through IRR, transitioning several filters from StreamBuffer to Stream body mode, adding terminal_streaming config to openai_responses_proxy, and implementing logical stream normalization in openai_stream_events.

Overall: Well-structured change with thorough test coverage across unit, integration, and inference fixture levels. The streaming dispatch guards (streamed_round_is_dispatchable, incomplete/failed stream protection) are sound. One control-flow issue in the error path.

Severity Count
Critical 0
Large 0
Medium 1

"openai_agentic_loop supports exactly one function call per round",
)?;
}
Ok(true)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] prepare_streamed_round returns Ok(true) after end_stream_with_error, which already sets ACTION_DONE and clears both call vectors. Returning true causes evaluate_loop_decision to run, re-enter the empty-tool-calls branch, call finalize_response_body (creating body bytes from response_object), and redundantly set ACTION_DONE again.

With logical_stream: true the body is overwritten by finalize_logical_stream, masking the issue. In a non-logical-stream configuration the serialized JSON leaks to the client after the SSE events.

Return Ok(false) after end_stream_with_error so the caller uses the early-return path (insert state, return Continue) instead of falling through to evaluate_loop_decision:

if state.tool_calls.len() > 1 {
    end_stream_with_error(
        ctx,
        state,
        "invalid_request_error",
        "openai_agentic_loop supports exactly one function call per round",
    )?;
    return Ok(false);
}

Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
The irr-terminal-streaming.yaml load_balancer cluster proxies to a
loopback endpoint (127.0.0.1:3001). Praxis main hardened cluster
endpoint validation to reject sensitive addresses unless
insecure_options.allow_private_endpoints is set, which broke the
all_example_configs_parse schema test under the praxis-main job.

Add the top-level insecure_options.allow_private_endpoints override,
matching every other example config that fronts local backends.

Signed-off-by: Sébastien Han <seb@redhat.com>
@leseb
leseb force-pushed the leseb/issue-706-implementation branch from 7e6a463 to 425f41e Compare September 1, 2026 09:23

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the checked-in agentic-loop example currently fails to parse because iterative_request_router rejects max_stream_response_bytes. this prevents the example from starting and causes the new integration, vLLM, and coverage jobs to fail before they exercise streaming. we need to remove or rename this setting to the router’s supported field, then rerun the new streaming coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

feat(openai): select terminal Responses streaming through IRR

4 participants