[bug] Responses conversation continuation drops materialized cross-format history

[Atharva-Kanherkar]

Symptom

A Responses client continuing a cross-format Chat conversation by conversation loses all prior turns. Continuing by previous_response_id keeps them.

Reproduction

Pinned at bfcd023c (current main, 2026-09-20), passthrough route from OpenAI Responses to an OpenAI Chat backend (format = "openai_chat") with a deterministic capture upstream. Seed then recall:

{"model":"switchyard/chat","input":"SEED","conversation":"conv_1"}
{"model":"switchyard/chat","input":"RECALL","conversation":"conv_1"}

The recall reaches the Chat backend with only RECALL. Control: same route, same seed, replace conversation with the returned previous_response_id. The backend then receives the seed user turn, seed assistant answer, and recall.

Result: history lost 5/5 with conversation, preserved 5/5 with previous_response_id.

Expected vs. actual

• Expected: both continuation mechanisms carry the same materialized history. PR fix(client): materialize cross-format Responses state #781 promises canonical history for cross-format stored Responses continuations.
• Actual: the conversation follow-up forwards only the new turn. The server returns HTTP 200 with a plausible but context-free answer.

Root cause

crates/libsy-llm-client/src/run.rs at this commit. remember_canonical_response (line 839) calls:

.remember(response_id, None, model, Some(history))   // line 863

The conversation id is hardcoded to None. remember_response (line 828) passes the real conversation, and native Responses recording passes both ids (line 718). The later conversation lookup misses, so routing silently continues without history.

Environment

• Switchyard version (or commit SHA): bfcd023cb6791efd8ac59d32afb5095c74fef1ff
• Python version: N/A
• OS / arch: macOS arm64
• Install path: source build
• Inbound format: Responses
• Backend: OpenAI Chat Completions (local deterministic capture)

Additional context

• Gap in fix(client): materialize cross-format Responses state #781 (merged 2026-09-18), which materializes cross-format state for previous_response_id only. PR fix(llm-client): keep Responses follow-ups on the provider holding their history #721 documents conversation as an equivalent supported continuation.
• Suggested fix: pass the request conversation id into remember_canonical_response and store the materialized history under both the response id and the conversation id, with buffered and streamed cross-format conversation tests.

cc @sabhatinas

[harshitwandhare]

Reproduced on main at 66bac290 with a unit test in run.rs that drives three turns on one conversation through remember_state_owner, stored_state_owner and continue_on. Turn 2 finds no stored state for conv_1, so this is still live.

One catch with the suggested fix. Passing the conversation id into remember_canonical_response gets turn 2 right, but StateOwners::remember writes the conversation key with entry().or_insert, so it keeps the first turn's history for good. With only that change applied, turn 3 goes out as:

left:  ["seed", "first answer", "third"]
right: ["seed", "first answer", "recall", "second answer", "third"]

The second exchange drops with no error, so the same symptom comes back one turn later. First write wins is right for provider-owned state, where the entry only pins routing. For materialized history the conversation entry has to move forward each turn, the way the response_id branch just above it already uses insert when materialized is true. With both changes the test passes, and cargo test -p switchyard-llm-client goes from 76 to 77 passed, 0 failed.

I only tested the buffered path. The streamed path calls the same remember_canonical_response, so it should need the same argument, but I have not run a streamed case.

@Atharva-Kanherkar are you planning to open the PR for this? If not, I'm happy to send it with the test.