feat(vision): add deepseek-v4-flash-vision-exp native image input (#242)

[demo-zexuan]

Summary

• Add deepseek-v4-flash-vision-exp model registration
• Send native image+text content blocks for vision-exp
• Keep vision proxy fallback for non-native models
• Update i18n/docs/config and diagnostics for multimodal content

Dependency Changes

• package.json dependencies/devDependencies: no direct version changes
• package-lock.json: transitive lockfile resolution updated (versions/resolved source refresh)

Testing

• npm run compile

Test Case: Identify and solve a linear algebra problem：

Closes #242

[Copilot]

Pull request overview

This PR adds a new experimental model deepseek-v4-flash-vision-exp that supports native multimodal (image + text) input, addressing issue #242. Instead of routing images through the existing vision-proxy pipeline, native-vision models send OpenAI-style content blocks (text parts and image_url data URLs) directly to the DeepSeek API. To support this, DeepSeekMessage.content is broadened from string to string | DeepSeekContentPart[], and the diagnostics/dump/classifier code paths that assumed a string are updated to extract text from the new array shape. The model is registered in the model registry, config schema, i18n, and README.

Changes:

• Register deepseek-v4-flash-vision-exp with a nativeImageInput capability, and build native content blocks in convertMessages while bypassing the vision proxy in prepareChatRequest.
• Broaden DeepSeekMessage.content to allow content-part arrays, and update classifier/dump/diagnostics/error-char counting to handle both string and array content.
• Update i18n, README (both locales), package.json, and NLS config for the new model; refresh package-lock.json transitive dev dependencies.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/types.ts	Adds content-part types and nativeImageInput capability flag
src/consts.ts	Registers the new native-vision model
src/provider/convert.ts	Builds native text/image content blocks for user messages
src/provider/request.ts	Bypasses vision proxy for native models; import block re-indented with spaces
src/provider/routing/classifier.ts	Extracts text from array content
src/provider/debug/dump.ts	Adds getDeepSeekMessageText helper for content extraction
src/provider/debug/diagnostics.ts	Adds toDiagnosticContentText; import block re-indented with spaces
src/client/error/index.ts	Counts chars across string/array content; import block re-indented with spaces
src/i18n.ts, package.nls*.json, package.json	Adds strings/config for the new model
README.md, README.zh-cn.md	Documents the new model
package-lock.json	Transitive dev-dependency lockfile refresh

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Vizards]

Hi @demo-zexuan — thank you for responding to this request and putting together a working implementation so quickly!

I’ve decided to use this PR as the foundation for adding native vision support. To keep this PR focused and complete, please limit its scope to the new Vision Exp model and the existing user-attachment vision flow. I may follow up with a stacked PR for images returned by agent tools, so there is no need to handle tool-result images in this PR.

Before the next review, could you please make the following changes?

1. Static checks and formatting

The approved CI workflow is currently failing at npm run format:check. The same failure is reproducible locally in these files:

• src/provider/content.ts
• src/provider/convert.ts
• src/provider/debug/diagnostics.ts
• src/provider/debug/dump.ts
• src/provider/request.ts

npm run lint passes, and npm run compile also passes locally, but the remote workflow stops after the formatting failure. Please run the formatter and verify that all of the following pass:

npm run lint
npm run format:check
npm run compile

2. Existing Copilot review

Thank you for following up on the Copilot review comments. The shared content helper introduced in src/provider/content.ts addresses the helper-duplication suggestion well.

Please review each Copilot comment individually. After the requested change has been implemented and verified, reply where useful and mark the corresponding review thread as Resolved. If a suggestion is not applicable or should not be implemented, briefly explain the reason in the thread before resolving it. Please do not mark a thread as resolved before the underlying issue has actually been addressed.

Some formatting threads are already marked as resolved, but the latest CI result still reports formatting problems, including in diagnostics.ts and request.ts. Please treat the formatter output as authoritative, clean up all five reported files, and verify the relevant review threads against the final code.

Once these changes are ready, I plan to run an Agent Peer Review over the complete implementation. There may be additional review feedback at that point, and I’d appreciate your help making any necessary adjustments and resolving the corresponding review threads after they are addressed.

3. Make Vision Exp the default vision route for Flash and Pro

For image requests made through DeepSeek V4 Flash or Pro, please replace oswe-vscode-prime with deepseek-v4-flash-vision-exp as the exact default vision model.

The intended behavior is:

• If the user explicitly configures a vision model, continue to respect that selection.
• Otherwise, use deepseek-v4-flash-vision-exp.
• Do not automatically fall back to OSWE or another arbitrary model.
• If Vision Exp cannot be found or its request fails, use the existing unavailable/failure notice and ask the user to configure a vision model manually.
• Explicitly selecting Vision Exp as the main chat model must continue to use native image input directly. Failures from its configured API endpoint should be reported directly, without falling back to the proxy path.

The current vision-model filtering excludes the entire deepseek vendor. Please make a narrow exception for deepseek-v4-flash-vision-exp, while continuing to exclude DeepSeek Flash and Pro so that they cannot recursively select themselves as vision proxies.

Please also include Vision Exp in the vision-model selector. In automatic mode, selection should require an exact Vision Exp match; avoid falling back to models[0] when it is unavailable.

4. Keep image token estimation conservative

The current implementation includes the full base64 image URL length in countMessageChars. That value is later used to update the shared adaptive charsPerToken ratio, so a native image request can distort token estimates for subsequent text requests.

For this initial implementation, please use a deliberately conservative and simple policy:

• Count each image as a fixed 384 tokens in provideTokenCount.
• Keep image tokens separate from text character estimation; the fixed image estimate must not be divided by the adaptive charsPerToken value.
• Do not include base64 image data in character counting.
• Skip the adaptive charsPerToken update for requests containing native images, because the returned prompt-token total does not provide enough information to separate text tokens from image tokens.

I can introduce dimension-aware image token calculation later if it becomes necessary.

5. Report native vision diagnostics accurately

The native path currently creates a VisionResolutionStats object containing only zero values, even when images are forwarded to the model.

Please keep this as a small, incremental diagnostics change rather than redesigning the diagnostics system. A suitable approach would be:

• Add an image-handling mode such as none, proxy, or native.
• For native requests, populate the existing input image counts instead of reporting zero.
• Add only the small number of native fields needed to distinguish forwarded and dropped images, for example:
  • forwardedImageParts
  • droppedImageParts
  • inputImageBytes
• Produce a trace along the lines of:
  vision mode=native inputImages=2 forwarded=2 dropped=0 bytes=...
• Do not include full base64 image URLs in metadata diagnostics. Use image metadata or a stable placeholder instead. Full image data may remain present only in the explicitly enabled verbose request dump.

Proxy-specific counters can remain unchanged and may be zero when the mode is native.

6. Repository hygiene

No dependency changes are required for this feature, so please:

• Revert the extension version change from 0.7.1 to 0.7.2 in package.json, while keeping the functional model and configuration additions.
• Restore package-lock.json to the base branch version, removing the unrelated transitive dependency upgrades, registry changes, and version bump.
• Revert the README changes from this PR. I’ll handle the final English and Chinese documentation updates separately in a follow-up PR.

7. Manual regression testing

Because of the evolving compatibility matrix across VS Code, GitHub Copilot, and the Language Model API, stable automated regression coverage is difficult for these scenarios.

Please perform a best-effort manual regression test using the latest stable VS Code release:

• Version: 1.134.0
• Commit: 110a328ea54b42367b803ec53ee0bf52

Only the following scenarios need to be covered in this PR:

1. The new deepseek-v4-flash-vision-exp model can respond successfully to a user prompt containing both image and text input.
2. The existing DeepSeek V4 Flash and Pro models can correctly forward an attached image to Vision Exp and receive a text description through the vision-proxy flow.

Please report the result of these two checks in the PR when the implementation is ready. I can perform the remaining compatibility and regression testing afterward.

Thanks again for getting the initial native vision path working so quickly. This gives the project a solid foundation to build on.

[demo-zexuan]

1. Repository hygiene

done

2. Report native vision diagnostics accurately

The relevant attribute fields have been supplemented in the class, and debug visibility is working properly.

Native Case

Proxy Case

The core changes are mainly within VisionResolutionStats.

The logic here is a bit messy… To be honest, it requires a major overhaul.

3. Trace

However, the time‑consumption during the proxy process here is slower than I expected... The time taken by the assembly tool here is incredibly high...

4. Keep image token estimation conservative

5. Make Vision Exp the default vision route for Flash and Pro

Adjustments have also been completed here. After several rounds of testing, the logic is currently working properly.

---

These are all the points we've noticed so far. We will need to conduct further reviews for more details. There are two aspects I haven't fully figured out yet:

1. Token statistics
2. Whether the latency introduced by the current vision model proxy poses an issue

@Vizards

[Copilot]

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

[Vizards]

Focused follow-up review against the routing and diagnostics behavior agreed for this PR. The core implementation is close; the inline comments identify the three remaining behavior gaps.

[Vizards]

Thank you for continuing to work through these follow-up details. If any part of the expected behavior is still unclear—especially the diagnostics sanitization or how its metadata representation should fit into the existing cache-trace logic—please complete the portions you are confident about and let me know once you are done. I can take over the remaining cleanup and final integration from there. There is no need to expand this PR into a broader diagnostics redesign just to resolve that uncertainty.

[leowork1273]

Always keeping an eye on these two experts and waiting for the update to be available in VSCode.😊

[Copilot]

🔵 Needs a closer look

It changes core request-building, token-accounting, and vision-routing logic across many files with no automated test coverage, so final human verification is warranted.

Review details

• Files reviewed: 21/21 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[demo-zexuan]

@Vizards I have improved it as much as possible. Please take some time to revise and adjust it, thank you 😊

[Vizards]

Thanks again for the quick work and for addressing the review feedback. Everything looks good on my side now, and I'll merge this shortly.