Skip to content

fix: set Content-Length on passthrough response body (empty non-streaming responses) - #308

Open
noyitz wants to merge 1 commit into
llm-d:mainfrom
noyitz:fix/response-content-length-passthrough
Open

fix: set Content-Length on passthrough response body (empty non-streaming responses)#308
noyitz wants to merge 1 commit into
llm-d:mainfrom
noyitz:fix/response-content-length-passthrough

Conversation

@noyitz

@noyitz noyitz commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #307.

Non-streaming responses were delivered downstream with an empty body (Content-Length: 0) whenever the response body was not mutated by a plugin (passthrough). The root cause is an asymmetry between the request and response handlers: the request handler always re-asserts Content-Length (including its non-mutated branch, with an explicit "always set Content-Length to inform Envoy of the body size that will follow" comment), but the response handler only did so when the body was mutated.

The buffered response body is re-emitted to Envoy as a streamed body replacement (AddStreamedResponseBody). In FULL_DUPLEX_STREAMED mode Envoy relies on the ext_proc to declare the size of the replaced body, so the passthrough branch — which omitted Content-Length — produced an empty body downstream.

Why streaming was unaffected: SSE responses have no upstream Content-Length and are streamed normally. Why mutated responses were unaffected: the bodyMutated branch already sets Content-Length from the re-serialized body.

Change

  • HandleResponseBody: add the symmetric else branch that sets Content-Length to len(responseBodyBytes) on the non-mutated (passthrough) body.
  • generatePassthroughResponseBodyResponse: set Content-Length before building the headers response (this helper is only reached from HandleResponseBody with the complete buffered body, so len(responseBodyBytes) is the full body).

Test plan

  • Updated TestHandleResponseBody_NoPlugins and TestHandleResponseBody_PluginNoBodyMutation to assert the passthrough Content-Length header (covers both passthrough entrypoints).
  • make test (race) green.
  • golangci-lint (v2.8.0) — 0 issues.
  • go vet / gofmt clean.

The response handler only re-asserted Content-Length when a plugin mutated
the body. The non-mutated (passthrough) path emitted the body via a streamed
replacement without declaring its size. In FULL_DUPLEX_STREAMED mode Envoy
relies on ext_proc to declare the body size that follows (as the request
handler already does), so non-streaming responses with an unmutated body were
delivered downstream with Content-Length 0 / an empty body. Streaming (SSE)
responses were unaffected.

Set Content-Length symmetrically on the passthrough path in HandleResponseBody
and generatePassthroughResponseBodyResponse.

Fixes llm-d#307

Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>
@github-actions github-actions Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Aug 31, 2026

@yossiovadia yossiovadia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independently verified the root cause — the asymmetry between request.go:119-120 (always sets Content-Length) and response.go (only sets it when bodyMutated == true) is confirmed in the code.

Fix is minimal and correct:

  • HandleResponseBody else branch mirrors the request handler exactly.
  • generatePassthroughResponseBodyResponse covers both call sites (no-plugins early return and JSON/SSE parse-failure fallback).
  • Tests assert Content-Length is present in the passthrough HeadersResponse (was previously empty). Sort helpers already handle SetHeaders ordering.

No issues found. LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Non-streaming responses return empty body: Content-Length not set on passthrough response path (asymmetric with request handler)

2 participants