Image generation via Codex gpt-image-2: generate_image tool plus /image composer command

[acoliver]

Problem

The agent cannot produce images. There is no image generation service, no tool, and no UI path. Workflows that need one (mockups for the GPUI views, icons for assets/, diagram illustrations for docs) currently mean leaving the app for an external tool.

Everything needed to add it already exists in the codebase except the generation itself:

• Agent tool pattern: src/agent/tools/ (activate_skill, edit_file, read_file, shell_exec, search, write_file) shows the shape: a ToolDefinition with JSON input_schema, an executor implementing ToolExecutor<McpToolContext>, and approval routing through ToolApprovalDecision/ToolCategory.
• Auth: the Codex OAuth stack is already live (codex auth presenter, e2e_codex_signin_device_code tests), which is exactly what the Codex image endpoint needs.
• Result plumbing: serdesAI core responses already model generated files (output_files, serdes-ai-core/src/messages/parts.rs:967), so a tool can hand the model an artifact reference rather than base64.

Reference implementation on disk

llxprt-code (~/projects/llxprt/branch-1/llxprt-code) built this exact feature; we can mirror its structure:

• packages/core/src/services/image/ImageGenerationService.ts: backend-neutral contracts. ImageGenerateRequest { prompt, model?, background?, quality?, size: auto|1024x1024|1024x1536|1536x1024, n?, sessionId? } mirrors the Codex gpt-image-2 generate endpoint shape. ImageResult { mimeType, encoding: base64|url, data, revisedPrompt? }. ImageGenerationError carries status/endpoint/truncated body for diagnostics.
• packages/core/src/services/image/imageCapability.ts: a single ImageOperationRunner ({ prompt, outputPath, inputPaths? } → bounded result with absolute/relative paths, mimeType, backend, provider, model) that the /image command, CLI image mode, and the generate_image tool all share. The capability result never carries base64.
• packages/providers/src/openai/codexImageBackend.ts: the concrete Codex OAuth backend that calls the generate endpoint and normalizes b64_json responses.
• imageOperationDispatch.ts: routes generate vs edit (edit = same runner with 1-5 inputPaths).

Prospective design

1. Service: src/services/image/ with the backend-neutral request/result/error types mirroring the llxprt contracts above. No fetch logic in the types module; backends implement an ImageGenerationBackend trait.
2. Backend: Codex adapter in the image service reusing the existing Codex OAuth token store. Model defaults to gpt-image-2; request shape follows the endpoint (prompt, quality, size, background, n). Response arrives as b64_json and is normalized to encoding: base64, then written straight to the output path.
3. Shared runner: one entry point run_image_operation(prompt, output_path, input_paths) used by both the tool and the composer command, returning only path/mime/model metadata. Edit mode activates when input_paths is non-empty (1-5 images).
4. Agent tool: generate_image in src/agent/tools/ following the existing tool shape. Schema: prompt (required), output_path (required, must end in .png), input_paths (optional array for edits). Approval: treat like a file write (it creates a file on disk). On success the tool returns the output file as a ToolReturnContent::Image/output_files entry so the model knows what it produced and can reference the path in later steps.
5. Composer command: /image <prompt> in the chat input generates to a per-conversation media dir (sibling of conversations dir, same location proposed for vision attachments in Image input (vision): let vision-capable models see images from chat, files, and tools #248) and renders a result chip with the path; clicking opens it in the system viewer. A /image edit <path> <prompt> form covers the edit case without new UI.
6. Settings: image model is selected independently of the chat profile via AppSettingsServiceImpl keys (image_provider, image_model), defaulting to the Codex provider with gpt-image-2. The settings screen gains an "Image generation" row showing provider/model and the Codex connection state.
7. Errors (fail fast): backend failures surface as tool errors carrying status and endpoint context, the same diagnostic shape as llxprt's ImageGenerationError. No silent fallbacks to a different provider.

Acceptance criteria

• With a connected Codex account, /image a gear icon, flat, 32px writes a PNG under the media dir and shows a chip in chat.
• The generate_image tool works inside a normal agent turn, requires approval, and the model receives the output path.
• Edit mode with an existing input image produces a modified image.
• No Codex token or image bytes appear in logs, events, or config.json.
• Clear error when Codex is not connected, pointing at the existing sign-in flow.

Open questions

• Do we need n > 1 and size/quality flags exposed in the composer command syntax, or tool-only for now?
• Should generated images also be attachable back into the conversation for vision models (Image input (vision): let vision-capable models see images from chat, files, and tools #248 interplay), i.e. render inline in chat history?