fix: set Content-Length on passthrough response body (empty non-streaming responses) - #308
Open
noyitz wants to merge 1 commit into
Open
fix: set Content-Length on passthrough response body (empty non-streaming responses)#308noyitz wants to merge 1 commit into
noyitz wants to merge 1 commit into
Conversation
The response handler only re-asserted Content-Length when a plugin mutated the body. The non-mutated (passthrough) path emitted the body via a streamed replacement without declaring its size. In FULL_DUPLEX_STREAMED mode Envoy relies on ext_proc to declare the body size that follows (as the request handler already does), so non-streaming responses with an unmutated body were delivered downstream with Content-Length 0 / an empty body. Streaming (SSE) responses were unaffected. Set Content-Length symmetrically on the passthrough path in HandleResponseBody and generatePassthroughResponseBodyResponse. Fixes llm-d#307 Signed-off-by: Noy Itzikowitz <nitzikow@redhat.com>
yossiovadia
approved these changes
Aug 31, 2026
yossiovadia
left a comment
Contributor
There was a problem hiding this comment.
Independently verified the root cause — the asymmetry between request.go:119-120 (always sets Content-Length) and response.go (only sets it when bodyMutated == true) is confirmed in the code.
Fix is minimal and correct:
HandleResponseBodyelse branch mirrors the request handler exactly.generatePassthroughResponseBodyResponsecovers both call sites (no-plugins early return and JSON/SSE parse-failure fallback).- Tests assert Content-Length is present in the passthrough
HeadersResponse(was previously empty). Sort helpers already handle SetHeaders ordering.
No issues found. LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #307.
Non-streaming responses were delivered downstream with an empty body (
Content-Length: 0) whenever the response body was not mutated by a plugin (passthrough). The root cause is an asymmetry between the request and response handlers: the request handler always re-assertsContent-Length(including its non-mutated branch, with an explicit "always set Content-Length to inform Envoy of the body size that will follow" comment), but the response handler only did so when the body was mutated.The buffered response body is re-emitted to Envoy as a streamed body replacement (
AddStreamedResponseBody). InFULL_DUPLEX_STREAMEDmode Envoy relies on the ext_proc to declare the size of the replaced body, so the passthrough branch — which omittedContent-Length— produced an empty body downstream.Why streaming was unaffected: SSE responses have no upstream
Content-Lengthand are streamed normally. Why mutated responses were unaffected: thebodyMutatedbranch already setsContent-Lengthfrom the re-serialized body.Change
HandleResponseBody: add the symmetricelsebranch that setsContent-Lengthtolen(responseBodyBytes)on the non-mutated (passthrough) body.generatePassthroughResponseBodyResponse: setContent-Lengthbefore building the headers response (this helper is only reached fromHandleResponseBodywith the complete buffered body, solen(responseBodyBytes)is the full body).Test plan
TestHandleResponseBody_NoPluginsandTestHandleResponseBody_PluginNoBodyMutationto assert the passthroughContent-Lengthheader (covers both passthrough entrypoints).make test(race) green.golangci-lint(v2.8.0) — 0 issues.go vet/gofmtclean.