Skip to content

🤖 fix: normalize mux-gateway file parts so image attachments are accepted - #4115

Open
Neppkun wants to merge 1 commit into
coder:mainfrom
Neppkun:fix_image_upload
Open

🤖 fix: normalize mux-gateway file parts so image attachments are accepted#4115
Neppkun wants to merge 1 commit into
coder:mainfrom
Neppkun:fix_image_upload

Conversation

@Neppkun

@Neppkun Neppkun commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Attaching an image to a chat message on a mux-gateway model fails with "invalid request", while text-only turns work. The AI SDK v7 upgrade changed how @ai-sdk/gateway serializes file parts on the wire, and the gateway server still validates the old shape. This PR adds a wire-level fetch shim that rewrites file parts back to the encoding the server accepts.

Background

#3707 bumped @ai-sdk/gateway from 3.0.x (spec v3) to 4.0.15 (spec v4). The two versions encode inline file data differently:

SDK file part data on the wire
3.x (spec v3) "data:image/png;base64,iVBOR…" — plain string
4.x (spec v4) { "type": "data", "data": "iVBOR…" } — tagged object (or { "type": "url", "url": … })

The request captured from a failing turn shows the v4 object shape, and the gateway server rejects it. (The server also still emits v2-style flat usage that gatewayStreamNormalization.ts patches around, so it lagging the SDK's wire protocol is consistent.)

Implementation

  • src/node/utils/gatewayFilePartNormalization.ts (new): wrapFetchWithGatewayFilePartNormalization inspects gateway POST bodies and rewrites {type:"data"}data:<mediaType>;base64,… (falling back to application/octet-stream, matching the 3.x SDK) and {type:"url"} → the URL string. Covers user/assistant file and reasoning-file parts and files nested in tool-result content output — the same set the SDK's own maybeEncodeFileParts touches.
  • Requests without file parts are forwarded with the original init untouched (cheap substring pre-check, no JSON round-trip); content-length is dropped when the body is rewritten.
  • providerModelFactory.ts: wraps the mux-gateway baseFetch with it as the innermost layer, so the existing Anthropic cache-control and auto-logout wrappers still apply on top.

The module doc marks the shim as removable once the gateway server accepts spec-v4 file parts.

Validation

  • Confirmed the encoding difference by diffing maybeEncodeFileParts between @ai-sdk/gateway@3.0.110 and @4.0.15.
  • Unit tests cover the data/url/string/unknown conversions, in-place prompt rewriting (including reasoning-file and tool-result nesting), and the fetch wrapper's pass-through cases (text-only, non-POST, non-string body, unparseable JSON).

Risks

Low and scoped to mux-gateway requests that contain file parts; text-only requests are byte-for-byte unchanged. If the gateway server is later upgraded to accept only the v4 shape, this shim would need to be removed — the module comment calls that out.


Generated with xum • Model: anthropic:claude-fable-5-1 • Thinking: medium • Cost: $4.15

@Neppkun

Neppkun commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 064fee5da8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant