feat(web): show inline previews for images agents refer to - #216
Merged
Conversation
An agent that took a screenshot could only hand the user a path. Paths in prose stayed plain text, markdown images rendered as a broken file:// img, and "Viewed image" tool rows showed a name box because Claude's Read, Codex's view_image and MCP screenshot tools only name the file or bury the bytes in the raw tool result. Files outside the project root could not be opened at all. Every surface now goes through one loader that fetches the file over the existing projects.readFile WebSocket RPC (so relay-paired and remote clients work unchanged) and shows a thumbnail with the file chip beneath it; a missing file leaves the chip exactly as before. The server serves a file outside the workspace root only when its bytes are a verified raster image. Tool rows read image blocks already stored on the tool result, or load the named path lazily when the row renders.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
vouch:trusted
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When an agent takes a screenshot, the user only gets a path. A path in prose stays plain text,
renders as a brokenfile://image, and "Viewed image" tool rows show a name box instead of a picture. Files saved outside the project root (temp dirs, where agents usually put screenshots) could not be opened at all.Fix
One loader turns any local image reference into a thumbnail, with the existing file chip kept underneath as the caption. A missing or unreadable file leaves the chip exactly as it looked before; there is no error state.
ChatMarkdownall render the thumbnail. Bare prose paths are detected only for unambiguous forms (drive, absolute,~/,./,../) and only once a block has settled, so a half-streamed path never triggers a fetch.Readof an image, Codexview_image, MCP screenshot tools) read the base64 image blocks already stored on the raw tool result, or lazily load the named path when the row renders in the virtualized list.projects.readFilenow serves a target outside the workspace root only when its bytes sniff as a raster image (PNG/JPEG/GIF/WebP/BMP/AVIF). Text, SVG, and mis-named files stay refused exactly as before.Bytes travel over the existing
projects.readFileWebSocket RPC and nothing else, so relay-paired (phonelink) and direct remote clients need no special handling; relay frame chunking already covers the base64 response.Tests
WorkspaceFileSystem.test.ts: outside-root PNG served, outside-root text rejected,.pngwith script bytes rejected.markdown-links.test.ts: bare image path detection and boundaries.session-logic.test.ts: path-only tool image, inline image blocks on a tool result.ChatMarkdown.browser.tsx: thumbnail from a mocked read, chip-only when the file is missing.Gates: fmt, lint, server/web typecheck, targeted unit suites, and the ChatMarkdown / MessagesTimeline / ChatView browser suites are green.