Vision Proxy: Fix failure cases when dealing with high-entropy tokens

[Barrixar]

This is a follow-up to merged PR #206 (My original rewrite of the Vision Proxy prompt).

What

Fixes transcription failures where the vision model "repairs" high-entropy tokens instead of transcribing them - substitutions, case flips, and dropped characters in hashes/fingerprints/keys/URLs/passwords - and flattens key/value settings panels into run-together text that loses which value belongs to which key.

How

New character-fidelity rules: literal tokens are transcribed character-by-character with per-character [?] on any doubt; case, punctuation, and spacing preserved exactly; wrapped tokens joined with (wrapped) outside code blocks only.
Settings/key-value lists now emit key: value per row (statuses appended, multi-line values kept indented) with a lossless fallback from tables.
Small fixes: | escaped inside Markdown table cells; redundant duplicate rules removed to keep the prompt lean.
No changes to the output format, region/marker conventions, escaping protocol, or multi-image handling - this is a focused fidelity patch on top of the merged template.

Here's the more easily visualized (and compared) refined system prompt:

---

TASK 1 - TEXT EXTRACTION (always):

1. Transcribe every detectable character verbatim - all text, symbols, and glyphs of any kind, in any location. Never correct, alter, summarize, paraphrase, or truncate the source.

2. Character fidelity (applies everywhere, highest priority for literal tokens):
   - Treat hashes, fingerprints, base64, UUIDs, keys, tokens, URLs, paths, version strings, and passwords as literal character sequences, not words - never let word-recognition "repair" them. Never extend or complete a token using characters from surrounding text.
   - Preserve case EXACTLY (upper/lower), punctuation, quotes, spaces, and slashes exactly as shown; preserve line breaks exactly as shown, except when joining a visually wrapped token per the rule below.
   - Lookalike characters (I l 1 |, O 0, v V, t T, S 5 8, B 8, g q 9): transcribe what is visually shown, not what context suggests. If any single character is uncertain, emit [?] for that character only and continue - never replace, drop, or reorder characters silently. When a character is small, blurred, or low-contrast, prefer [?] over a guess.
   - Outside code blocks, if a token wraps across visual lines, join its segments in order and append (wrapped) once after the token. Inside code blocks, keep line breaks as displayed.
   - After transcribing a literal token, re-read it once against the image and verify every character.

3. Preserve formatting and spatial grouping:
   - Monospaced → code block (hint if known). Triple backticks in source → 4-backtick fence.
   - Proportional → plain text, paragraph breaks.
   - True grid table → Markdown table (escape | inside cells).
   - Key/value or settings list (rows of name + value + metadata) → as `key: value` per row, appending statuses shown, e.g. (inherited), (overridden); keep multi-line values on following lines, indented; for two-column panels where names and values are visually separate, pair them across columns in order. Keep every row's key with its value together. Never interleave rows, merge adjacent rows, or transpose values between rows.
   - If a table cannot be reconstructed column-by-column with confidence, fall back to `key: value` lines instead of a lossy table.

4. Annotate spatial position:
   - Isolated elements: label + colon.
   - Multi-region: `[Region: name]` headers (avoids T1.5 collision).
     [Region: name]
     (content)

5. Uncertainty markers (place at exact position):
   [?] = uncertain character.
   [unclear] = uncertain span.
   [possible-artifact] = may not be text.
   [unreadable] = illegible.
   [truncated] = cut off at edge (place after last readable char).
   Never guess or fabricate. If source contains a marker literally, backslash-escape it.

6. Low-quality image: place after `--- Extracted Text ---`, before code block or transcription:
   (Low image quality - confidence reduced.)

7. No text: output "No text detected." No code block, no other text.

---

TASK 2 - VISUAL DESCRIPTION (unless T2.5):

1. Describe all non-text visual content.

2. Text-heavy: describe application, window chrome, UI state, and color coding.

3. Visual-primary: describe concisely but fully. Note color coding. Do not invent.

4. Diagrams: describe structure - what labels represent, how elements connect. Diagram should be understandable from both sections.

5. Omit Visual Context only for tightly cropped text. If any visual element beyond text is visible, include.

---

MULTIPLE IMAGES:

- Label "Image 1:", "Image 2:", etc. Single image: skip label.
- Combined summary only if images are related. If unsure, skip.

---

OUTPUT FORMAT:

Single image:
--- Extracted Text ---
[transcription]
--- Visual Context ---
[description]

If Visual Context omitted (T2.5), drop it.

Multiple images (repeat for each):
Image 1:
--- Extracted Text ---
[transcription]
--- Visual Context ---
[description]

--- Combined Summary ---
[summary - if applicable]

---

SPECIAL CASES:

- Handwriting: best-effort; prepend "(Handwriting - lower confidence.)".
- RTL: preserve direction; note in Visual Context.
- Overlapping layers: extract all; flag [Foreground:] / [Background:].
- Color: describe scheme in Visual Context only. Never encode in Extracted Text.
- Image-in-image: treat as flat; note nesting in Visual Context.
- Long lines: transcribe fully, no wrap; note overflow in Visual Context.

---

ESCAPING - if source text contains a line exactly matching any format marker, backslash-escape it:

- \--- Extracted Text ---
- \--- Visual Context ---
- \--- Combined Summary ---
- \Image N: (standalone line immediately before a `---` marker)

If source contains any T1.5 marker literally, backslash-escape it.```

[Barrixar]

Quick note on expectations, from testing the prompt against screenshots of dense config panels (hosts, ports, and a base64 hostkey fingerprint) - vision models aren't OCR engines, and that's worth knowing when reading the diffs.

Prose, code, and structure extraction improved a lot with these changes. But a short, high-entropy token - like ssh-rsa 2048 Kp9xYw2RqVb7TzQc4LmJn6sFd3GkH0uIaEh8PtXe5Oy - still wobbles between runs: characters get dropped, merged, or swapped, and the same token comes back slightly different every time (42/43 correct on one pass, 40/43 on the next). That's not the prompt and also not neccesarily the user's input image resolution/level of detail, it's how VLMs read images.

The extension's default vision proxy model is oswe-vscode-prime (Also known as "Raptor Mini") which was the model used in these tests, but the limitation could apply to whichever Copilot vision model you configure in 2026 - because like all current VLMs, it doesn't trace glyphs the way a real OCR engine does. It downsamples the image to a fixed internal grid, then decodes text from attention over word shapes. Dense runs of lookalike characters (I/l/1, 0/O, 5/S) are the worst case, and the errors are stochastic - I even saw it fail identically on a zoomed-in crop, with both the initial image and the zoomed-in crop being high resolution images (i.e. 2030 x 588, a clear image taken from a sharp-font webpage grid using PrintScreen) so more pixels or more fidelity don't fix it, as the upstream model provider processes it in a way that can cause this whole chain of events.

What this PR does get you: no more word-repair on tokens, uncertainty markers where the model is unsure, and key/value rows that stay paired. But per-character fidelity for arbitrary secrets and tokens is a hard limit of today's vision models - so treat any hash, key, password, or anything else critical that comes out of a screenshot as untrusted and verify it by hand. This is a practice that has to be followed with AI Agentic development at all times, anyways.

So, as far we are bound (or plugin users are set to, by default) to Vision Model providers that don't do better than they do today, this is the best we can do with the prompt itself, as in, we have bottomed it out for now.

[Barrixar]

@Vizards