Non-streaming responses return empty body: Content-Length not set on passthrough response path (asymmetric with request handler)

[noyitz]

Summary

Non-streaming responses are delivered downstream with an empty body (Content-Length: 0) whenever the response body is not mutated by a plugin (passthrough). Streaming (SSE) responses are unaffected. The root cause is an asymmetry between the request and response handlers: the request handler always re-asserts Content-Length, but the response handler only does so when the body is mutated.

Root cause

Request handler always sets Content-Length — pkg/handlers/request.go (non-mutated branch):

} else {
    // Always set Content-Length to inform Envoy of the body size that will follow
    reqCtx.Request.SetHeader(contentLengthHeader, strconv.Itoa(len(requestBodyBytes)))
}

Response handler only sets it when the body is mutated — pkg/handlers/response.go, HandleResponseBody:

bodyMutated := reqCtx.Response.BodyMutated()
if bodyMutated {
    mutatedBytes, _ = json.Marshal(reqCtx.Response.Body)
    reqCtx.Response.SetHeader(contentLengthHeader, strconv.Itoa(len(mutatedBytes)))
}
// <-- no else: Content-Length is never re-asserted for the passthrough body
...
if bodyMutated {
    ret = envoy.AddStreamedResponseBody(ret, mutatedBytes)
} else {
    ret = envoy.AddStreamedResponseBody(ret, responseBodyBytes) // body emitted, but no Content-Length declared
}

The buffered response body is re-emitted to Envoy as a streamed body replacement (AddStreamedResponseBody → BodyMutation_StreamedResponse). In FULL_DUPLEX_STREAMED mode Envoy relies on the ext_proc to declare the size of the replaced body — exactly what the request handler documents. The passthrough response branch omits it, so the downstream response is delivered with an empty body. The same gap exists in generatePassthroughResponseBodyResponse (the no-plugins early return and the JSON/SSE parse-failure fallback).

Why streaming is unaffected and non-streaming breaks

• Non-streaming, no body mutation (e.g. an OpenAI-compatible upstream where response translation is a no-op → bodyMutated == false): passthrough branch runs, no Content-Length is declared → empty 200.
• Streaming (SSE): the upstream response has no Content-Length to begin with and is streamed; unaffected.
• Mutated responses (a translator re-serializes the body → bodyMutated == true): Content-Length is set at line ~103, so these are fine.

HandleResponseBody is only invoked with the complete buffered body at EndOfStream (pkg/handlers/server.go, gated by Profile.NeedsResponseBuffering), so len(responseBodyBytes) is always the full body in every path inside that function — including both generatePassthroughResponseBodyResponse call sites.

Impact

Silent data loss for clients: a non-streaming request returns 200 OK with no body and no error, for any deployment whose response is buffered (a response plugin chain is configured) and whose body is not mutated.

Proposed fix

Make the response handler symmetric with the request handler — set Content-Length on the non-mutated (passthrough) response path as well:

• add an else branch in HandleResponseBody that sets Content-Length to len(responseBodyBytes);
• set Content-Length in generatePassthroughResponseBodyResponse before building the headers response.

PR to follow.

Environment

• pkg/handlers/response.go on main (also present in the v0.1.0-rc.4 line).
• Reproduces with an OpenAI-compatible upstream and a response plugin chain configured (so responses are buffered), sending a non-streaming (stream: false) chat completion.