Investigate inline snapshot rendering in capable terminals

[obj-p]

Motivation

preview snapshot currently writes a PNG to disk and prints the path. For users on terminals that support inline images, we could render the snapshot directly in the terminal after (or instead of) writing the file — a much tighter feedback loop for iterating on UI.

Candidate protocols

• iTerm2 inline images — base64-encoded OSC 1337 escape (ESC ] 1337 ; File=...). Widely supported by iTerm2, WezTerm.
• Kitty graphics protocol — chunked OSC escape with broader capabilities; supported by Kitty, WezTerm, Ghostty.
• Sixel — older standard; supported by mlterm, foot, some xterm builds.

Open questions

• How do we detect terminal capability? ($TERM_PROGRAM, $TERM, $KITTY_WINDOW_ID, terminfo queries via XTGETTCAP?)
• Opt-in flag, opt-out flag, or auto-detect with override?
• Behavior when piping stdout (e.g. preview snapshot | jq) — should suppress images.
• Behavior in --json mode — keep path-only, or embed base64?
• Downscale large snapshots before emitting so we don't blow up the scrollback? What's the right max width?
• Does this fit in PreviewsCLI directly, or warrant a small helper module?

Acceptance criteria (tentative)

• Capability detection for at least iTerm2 + Kitty
• Graceful fallback to path-only output on unsupported terminals or non-TTY stdout
• Does not interfere with --json / scripting workflows
• Covered by tests (capability detection at minimum; rendering likely needs a manual checklist)

[obj-p]

Proposed plan (draft — seeking consensus before work starts)

Drafted in worktree worktree-issue-125-inline-snapshots. Posting here so decisions are reviewable before code.

1. Current state (ground truth)

• Sources/PreviewsCLI/SnapshotCommand.swift — the PNG/JPEG is decoded from the daemon response at lines 328–350. The "write file + print path" site is handleSnapshotResponse lines 333–346. The --json branch is lines 335–344; the bare-path branch is line 345. Data and mimeType are both in scope at the call site — that is the single hook point we need.
• Sources/PreviewsCLI/MCPContentHelpers.swift:54 — shared emitJSON helper; inline output should follow the same "print once, no trailing chatter" style.
• Tests/PreviewsCLITests/ has StallTimerTests.swift, VariantsExitCodeTests.swift — precedent for CLI-internal unit tests linked directly against the PreviewsCLI target (Package.swift ~line 87–89).
• No existing terminal/escape-sequence code in the tree (no hits for OSC, iTerm, Kitty, Sixel, 1337).

2. Implementation plan (7 steps)

Step 1 — New module Sources/PreviewsCLI/TerminalImages/ containing:

• TerminalImages.swift — renderInline(data:mimeType:mode:env:stdoutIsTTY:) -> InlineDecision returning .emit(bytes) or .skip(reason).
• TerminalCapability.swift — enum { iTerm2, Kitty, unsupported } + detect(env:).
• ITerm2Encoder.swift — OSC 1337 File=inline=1;size=N;width=auto;height=auto:<base64> framing, BEL-terminated.
• DownscaleCG.swift — CGImageSourceCreateThumbnailAtIndex (CoreGraphics/ImageIO already transitively available).
• Environment protocol (term, termProgram, tmux, kittyWindowID, previewsmcpInline) with ProcessEnvironment impl + MockEnvironment for tests.

Rationale for new subdirectory (vs. extension): encoder bytes, detection, and downscaling are a cohesive unit with their own tests; nothing else in PreviewsCLI calls them.

Step 2 — Wire into SnapshotCommand.handleSnapshotResponse (line 332–350). After try data.write(to: outputURL) and before the if json branch, call TerminalImages.renderInline(...). Add --inline <auto|always|never> (default auto). Honor PREVIEWSMCP_INLINE=0|1|auto.

Step 3 — Behavior matrix (default auto with override). Opt-out via --inline never or PREVIEWSMCP_INLINE=0; force via --inline always. Rationale: the feature is tighter feedback; requiring flag discovery defeats it. --json and piped output keep current behavior.

case	write file	print path	inline image
TTY + supported + no --json	yes	yes (after image)	yes
TTY + unsupported	yes	yes	no
non-TTY (pipe/redirect)	yes	yes	no
--json (any)	yes	no (JSON has outputPath)	no
--inline always + non-TTY	yes	yes	yes (override; stderr warn)
--inline never	yes	yes	no

Ordering in the TTY-supported case: image bytes first, newline, then path — keeps the path on the last line for copy/paste.

Step 4 — Capability detection.

• TERM_PROGRAM == "iTerm.app" OR LC_TERMINAL == "iTerm2" OR TERM_PROGRAM == "WezTerm" → iTerm2.
• TERM == "xterm-kitty" OR KITTY_WINDOW_ID set OR TERM_PROGRAM == "ghostty" → Kitty.
• Else → unsupported.
• No XTGETTCAP probing in v1 (requires async tty read with timeout).
• If TMUX is set, classify unsupported in v1 and emit one stderr hint: note: running inside tmux; see docs for enabling passthrough. Do not wrap in \ePtmux;...\e\\ — that only works with allow-passthrough on and otherwise just prints garbage.

Step 5 — Protocol scope: iTerm2 only in v1. Covers iTerm2, WezTerm, and most macOS devs; encoder is ~15 lines. Pushing back on the acceptance criterion "iTerm2 + Kitty minimum" — Kitty's chunked protocol is ~10× more code (chunk size limits, control/data/end markers, a=T,f=100 params) and the macOS audience is small. Land iTerm2 now; file a follow-up for Kitty.

Step 6 — Downscale. Threshold pixel width > 1200 → resize to width 1000 via CGImageSourceCreateThumbnailAtIndex with kCGImageSourceThumbnailMaxPixelSize=1000 and kCGImageSourceCreateThumbnailFromImageAlways=true, re-encoded to PNG for inline display only. The on-disk file at --output stays full resolution.

Step 7 — Tests — new Tests/PreviewsCLITests/TerminalImagesTests.swift:

• Golden-byte test for iTerm2 encoder: fixed 4-byte input → expected exact \u{1B}]1337;File=inline=1;size=4;...\u{07} string (base64 is deterministic).
• Capability detection table tests via MockEnvironment: iTerm, WezTerm, Ghostty, Kitty, bare xterm-256color, TMUX set → expected capability.
• renderInline decision table: (stdoutIsTTY, capability, mode, jsonFlag) → .emit/.skip matching the Step 3 matrix.
• Rendering itself is not asserted in tests — manual only.

3. Verification

Does the current dev session (the one drafting this plan) support inline rendering? No, and it can't be determined from inside the session. Two independent blockers:

1. The session is an SSH connection — the outer terminal lives on the local client machine, invisible to any server-side probe.
2. tmux allow-passthrough is off — even if the local terminal is iTerm2/WezTerm, OSC 1337 bytes get eaten by tmux.

To verify after shipping, do one of:

• Path A (cleanest): run previewsmcp preview snapshot <file> in a local iTerm2/WezTerm window (no SSH, no tmux). Expect inline render.
• Path B (stay in tmux): tmux set -g allow-passthrough on, then from a fresh local terminal (not the SSH session) run printf '%s=%s\n' TERM \"\$TERM\" TERM_PROGRAM \"\$TERM_PROGRAM\" LC_TERMINAL \"\$LC_TERMINAL\" KITTY_WINDOW_ID \"\$KITTY_WINDOW_ID\" TMUX \"\$TMUX\" so we can confirm detection classifies it correctly. v1 still won't render inside tmux, but the probe validates detection.

Manual test checklist (once landed):

1. Local iTerm2, no tmux: previewsmcp preview snapshot Foo.swift → image + path.
2. Local iTerm2, no tmux: ... | cat → path only, no escape bytes.
3. Local iTerm2: ... --json → JSON only.
4. Local iTerm2: ... --inline never → path only.
5. Local Terminal.app: ... → path only (unsupported fallback).
6. Inside tmux, passthrough off: ... → path + stderr hint, no garbage.
7. Large preview (width 2400): downscale kicks in; image renders; on-disk file stays full-res.

4. Tradeoffs + open questions

• Acceptance-criteria pushback: issue says "iTerm2 + Kitty minimum"; proposing iTerm2-only v1, Kitty as follow-up.
• tmux passthrough wrapping: not implementing in v1. DCS-wrapping only works when the user already set allow-passthrough on; otherwise behavior is identical to unwrapped. Docs note is cheaper than code that no-ops in the failure mode.
• Downscale threshold (1200/1000): arbitrary. Tune after seeing typical preview dimensions.
• XTGETTCAP capability probe: deliberately skipped. Revisit only if env-based detection misclassifies real terminals.
• Low confidence: whether Ghostty's iTerm2-protocol support is actually gated on TERM_PROGRAM=ghostty — docs say yes, not verified.

---

Consensus check — please confirm or amend before work starts:

• (a) iTerm2-only v1 (defer Kitty + Sixel)
• (b) auto default with --inline/PREVIEWSMCP_INLINE override
• (c) No tmux DCS wrapping in v1 (docs-only guidance)
• (d) Downscale threshold 1200px → target 1000px
• (e) Inline-before-path ordering in TTY output

[obj-p]

Amendment: tmux support in v1

Revising the prior plan. The "no tmux DCS wrapping in v1 (docs-only)" decision was based on a wrong premise — that wrapping is a no-op when passthrough is off. In fact the failure mode is asymmetric: without wrapping, tmux eats the escape even when allow-passthrough is on.

tmux passthrough	wrapped	result
off	no	tmux eats it (path-only fallback, expected)
off	yes	tmux discards passthrough (same fallback)
on	no	tmux eats it silently (broken feature)
on	yes	image renders in outer terminal

So wrapping is a prerequisite for tmux users, not an optimization. Adding it to v1.

Plan changes

Step 4 (Capability detection) — revised TMUX handling:

• Drop the "classify TMUX as unsupported in v1" rule.
• Add a one-time check: if $TMUX is set, run tmux show-options -v -g allow-passthrough with a short timeout.
  • on → flag needsTmuxWrap = true, proceed with detection as normal.
  • off (or command fails / times out) → classify unsupported for this invocation; emit one-line stderr hint: note: running inside tmux; enable with tmux set -g allow-passthrough on.

New Step 4b (tmux DCS wrapper) — new file Sources/PreviewsCLI/TerminalImages/TmuxPassthrough.swift:

• Function wrap(_ inner: Data) -> Data emitting \ePtmux; + inner with every \x1B doubled to \x1B\x1B + \e\\.
• Pure byte transform, no state. Golden-byte unit test asserting the exact output for a fixed input.

Step 2 (SnapshotCommand wiring) — minor tweak: TerminalImages.renderInline(...) returns bytes already wrapped if needsTmuxWrap was set on the capability decision.

Step 7 (Tests) — added cases:

• TmuxPassthroughTests: golden-byte test; double-ESC invariant over a mixed-byte input containing \x1B, \x07, and a base64 payload.
• TerminalImagesTests.renderInline decision table gains rows for (TMUX set, allow-passthrough=on, iTerm2) → .emit(wrapped) and (TMUX set, allow-passthrough=off) → .skip(.tmuxPassthroughOff) with expected stderr hint.
• Capability detection tests add a MockTmuxProbe seam so the tmux show-options call can be stubbed.

Manual checklist additions:
8. Inside tmux, allow-passthrough off, local iTerm2: snapshot → path + one-line stderr hint; no garbage bytes in pane.
9. Inside tmux, allow-passthrough on, local iTerm2: snapshot → image renders in pane; path printed on line after.
10. Inside tmux, allow-passthrough on, local Terminal.app (unsupported outer terminal): snapshot → path only (no wrap attempted because outer terminal classification is still unsupported).

Known tmux caveats (documented, not fixed in v1)

• Images don't persist in tmux scrollback — tmux redraws from a text buffer, so scroll-back past the line shows blank space. Fresh renders work.
• Older tmux versions had a passthrough payload cap (~1MB); downscale to 1000px keeps base64 PNGs well under that.
• Cursor advance after the image is approximate — may leave a one-line gap. Acceptable for v1.
• No attempt to detect the outer terminal from inside tmux via XTGETTCAP in v1; env-based classification only, using whatever TERM_PROGRAM / LC_TERMINAL / etc. leaked through (tmux may strip these on newer versions — in that case we fall back to unsupported, which is the safe default).

Revised consensus check

Replaces item (c) in the prior comment:

• (a) iTerm2-only v1 (defer Kitty + Sixel) — unchanged
• (b) auto default with --inline/PREVIEWSMCP_INLINE override — unchanged
• (c) Include tmux DCS passthrough wrapping in v1, gated on a tmux show-options probe; stderr hint when passthrough is off
• (d) Downscale threshold 1200px → target 1000px — unchanged
• (e) Inline-before-path ordering in TTY output — unchanged