Skip to content

fix(openai-shim): include text with image-only content#1410

Open
MikeTomlin19 wants to merge 1 commit into
Gitlawb:mainfrom
MikeTomlin19:fix/issue-425-image-text-part
Open

fix(openai-shim): include text with image-only content#1410
MikeTomlin19 wants to merge 1 commit into
Gitlawb:mainfrom
MikeTomlin19:fix/issue-425-image-text-part

Conversation

@MikeTomlin19
Copy link
Copy Markdown
Contributor

@MikeTomlin19 MikeTomlin19 commented May 28, 2026

Summary

  • Ensure OpenAI-compatible multipart image payloads always include a non-empty text part.
  • Preserve existing mixed text+image payloads while adding coverage for image-only user and tool-result paths.

Why

Some OpenAI-compatible providers, including the Xiaomi MiMo/OpenGateway reports discussed around #425, reject image requests with text is not set when the content array contains only image_url parts.

Refs #425.

This is intentionally scoped to the MiMo/OpenGateway image-capable compatibility case. It does not attempt to solve text-only provider image handling, which is tracked separately by #1382/#1391.

Test plan

  • bun test src/services/api/openaiShim.test.ts
  • bun test src/services/api/openaiShim.test.ts src/utils/model/modelOptions.xiaomi-mimo.test.ts src/services/api/providerConfig.local.test.ts
  • bun run build
  • bun run smoke
  • git diff --check
  • GitHub web check
  • GitHub smoke-and-tests reached the full suite and failed only on the same 4 unrelated tests already failing on latest main PR Checks run 26548651163
  • bun test locally (3005 pass / 10 fail; failures are unrelated existing attribution, auto-compact, breaker metadata, and provider model env tests)
  • [blocked] bun run typecheck (repo-wide generated/missing-module/type errors unrelated to this patch; first failure is src/QueryEngine.ts missing ./services/compact/snipProjection.js)

@Meetpatel006
Copy link
Copy Markdown
Contributor

can you check for the ci faling issue

@MikeTomlin19
Copy link
Copy Markdown
Contributor Author

Checked it. The failing check is the full smoke-and-tests suite, not the targeted image payload coverage from this PR.

The run failed on these 4 tests:

  • getAttributionTexts > preserves includeCoAuthoredBy true as an explicit old-default opt-in
  • active auto-compact cooldown blocks before model call with cooldown guidance
  • auto-compact cooldown tracking is carried into the next query call
  • breaker metadata tracking callback publishes a fresh object

Those are outside this PR's OpenAI shim/image-only content path. This PR's targeted tests and build/smoke passed locally, and the separate web check is green here. I also see main had a PR Checks failure around the same window, then a later main PR Checks run is green, so this looks like stale base/CI state rather than something introduced by this patch.

A rerun after the current main state should be the right next check.

@MikeTomlin19 MikeTomlin19 force-pushed the fix/issue-425-image-text-part branch from 2ca5a2a to 94f615c Compare May 28, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants