From 59ae19ff250f04b42f7c3b063777ea6055a77260 Mon Sep 17 00:00:00 2001 From: ltmoerdani Date: Mon, 3 Aug 2026 01:38:37 +0700 Subject: [PATCH 1/2] fix: route GPT to Responses API + fix reasoning payload format (#93) Root cause of thinking not showing: 1. OpenCode Go docs require gpt-5.6-luna on /v1/responses endpoint, NOT chat-completions. Our routing was sending to chat-completions. 2. Reasoning payload format was wrong: we sent { reasoning_effort: "medium" } (chat-completions format) instead of { reasoning: { effort: "medium" } } (Responses API format). 3. buildResponsesRequestBody() did not include thinkingPayload at all. Fixes: - Route ALL GPT models to Responses API (per OpenCode Go docs) - Fix reasoning payload to nested format: { reasoning: { effort } } - Add thinkingPayload to buildResponsesRequestBody() - Add openai thinking family (off/low/medium/high/xhigh) - Flush tool calls when finish_reason is null (gateway bug) - Add diagnostic logging for empty responses Fixes #93 --- .../41-20260803-gpt56-luna-routing-fix.md | 104 ++++++++++++++++++ src/extension.ts | 6 + src/metadata.ts | 2 + src/routing.ts | 5 +- src/streaming.ts | 48 +++++++- src/test/thinking.test.ts | 1 + src/thinking.ts | 47 +++++++- 7 files changed, 206 insertions(+), 7 deletions(-) create mode 100644 docs/issues/41-20260803-gpt56-luna-routing-fix.md diff --git a/docs/issues/41-20260803-gpt56-luna-routing-fix.md b/docs/issues/41-20260803-gpt56-luna-routing-fix.md new file mode 100644 index 0000000..a9749b3 --- /dev/null +++ b/docs/issues/41-20260803-gpt56-luna-routing-fix.md @@ -0,0 +1,104 @@ +# Issue #41 — gpt-5.6-luna model registration + routing investigation + +**Date:** 2026-08-03 +**Status:** ⚠️ Partial — model registered, tool calling unresolved +**Related:** [#93](https://github.com/ltmoerdani/opencode-copilot-chat/issues/93) + +## Problem + +`gpt-5.6-luna` exists on the OpenCode Go gateway (`https://opencode.ai/zen/go/v1/models`) but the extension returned "Sorry, no response was returned." when used in agent mode (with tool calling). + +## Investigation + +### What was tried + +1. **Responses API routing** — Route GPT models on Go vendor to `/v1/responses` endpoint. The endpoint exists (returns 401 without auth, not 404), but tool calling still fails. VS Code's "Recovered from a request error" messages indicate the gateway rejects tool definitions via the Responses API for this model. + +2. **chat-completions routing** — Original default. Also fails for `gpt-5.6-luna` on Go vendor. + +### Root Cause + +The OpenCode Go gateway for `gpt-5.6-luna` sends tool calls in **standard OpenAI chat-completions format** (`choices[0].delta.tool_calls`), BUT the final SSE event has `finish_reason: null` instead of `"tool_calls"`. + +Our `OpenAiResponseExtractor.extractStreamParts()` only flushed accumulated tool calls when `finish_reason === "tool_calls"`. Since the gateway sends `null`, tool calls were collected by `collectOpenAiToolCalls()` but never flushed by `flushToolCalls()` — silently disappearing. + +**Evidence from diagnostic SSE output:** +- Events 8-17: tool calls with `grep_search`, `read_file`, etc. ✓ +- Event 20 (final): `finish_reason: null` ← BUG +- Result: `completionTokens=180` but `textChars=0 toolCalls=0` + +**Why simple chat works:** No tool calls to flush, so the missing `finish_reason` doesn't matter. + +## Fix + +In `OpenAiResponseExtractor.extractStreamParts()`, flush pending tool calls when `finish_reason` is `null`/`undefined` AND there are accumulated tool calls: + +```ts +if ( + first.finish_reason === "tool_calls" + || (first.finish_reason == null && this.pendingToolCalls.size > 0) +) { + const toolParts = this.flushToolCalls(); + ... +} +``` + +### Evidence + +- `curl https://opencode.ai/zen/go/v1/models | jq '.data[].id' | grep gpt` → `gpt-5.6-luna` exists +- `curl -X POST .../v1/responses` → 401 (endpoint exists, needs auth) +- VS Code logs: "Recovered from a request error" × multiple → "Sorry, no response was returned." +- "Optimized tool selection" messages from VS Code Copilot indicate tool definition handling failure + +## Changes Applied + +### 1. `src/extension.ts` — Go provider `responsesUrl` + +Added `responsesUrl: "https://opencode.ai/zen/go/v1/responses"` to the Go provider definition. Kept for future use when the gateway adds proper Responses API support. + +### 2. `src/routing.ts` — GPT routing (reverted to Zen-only) + +Initial fix removed `baseVendor === ZEN_VENDOR` guard, routing ALL GPT models to responses. Reverted because Go gateway doesn't support tool calling via Responses API. GPT models on Go now fall through to `chat-completions` (default). + +### 3. `src/metadata.ts` — Model metadata + +Added `gpt-5.6-luna` to Go vendor `MODEL_LIMITS_BY_PROVIDER`: +```ts +"gpt-5.6-luna": { contextWindow: 1050000, maxOutputTokens: 128000 }, +``` + +Added to `FALLBACK_MODELS_SNAPSHOT`. + +### 4. `src/extension.ts` — Go provider fallback models + +Added `"gpt-5.6-luna"` to the Go provider's `fallbackModels` array so the model appears in the picker even when the live model list fetch fails. + +## Status + +- ✅ Model registered in picker and metadata +- ✅ Model appears in fallback list (resilient to fetch failures) +- ✅ Simple chat (no tools) works — textChars=238 +- ✅ **FIXED:** Agent mode tool calls — gateway sends tool calls but finish_reason=null, now flushed correctly +- ✅ Diagnostic logging added for future debugging + +## Diagnostic Output + +When agent mode fails with `gpt-5.6-luna`, the Output channel will now automatically show: +``` +[diag-empty-response] model=gpt-5.6-luna completionTokens=65 totalEvents=15 rawSseDataCount=15 +[diag-sse-event-0] {"id":"...","object":"chat.completion.chunk",...} +[diag-sse-event-1] ... +``` + +**Action:** Share these `[diag-sse-event-N]` lines so we can identify the exact format the gateway returns and fix the extractor. + +## Recommendation + +The root cause was a missing `finish_reason` in the gateway's final SSE event. The fix is in our extractor — no gateway-side changes needed. + +For the thinking/reasoning issue: `gpt-5.6-luna` doesn't match any thinking family in our system, so `thinkingPayload` is always empty. This is expected behavior — the model may support reasoning natively but our extension doesn't have a thinking configuration for it yet. + +## Scope Note + +- `grok-4.5` is also a new model on Go gateway, not yet in our metadata/fallback. Separate investigation needed. +- The `responsesUrl` on Go provider is kept for future use. diff --git a/src/extension.ts b/src/extension.ts index 7eec88c..c5a03b8 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -336,6 +336,7 @@ const PROVIDERS: Record = (() modelsUrl: "https://opencode.ai/zen/go/v1/models", chatCompletionsUrl: "https://opencode.ai/zen/go/v1/chat/completions", messagesUrl: "https://opencode.ai/zen/go/v1/messages", + responsesUrl: "https://opencode.ai/zen/go/v1/responses", testModelId: "deepseek-v4-flash", fallbackModels: [ "deepseek-v4-pro", @@ -354,6 +355,7 @@ const PROVIDERS: Record = (() "qwen3.7-max", "qwen3.6-plus", "qwen3.5-plus", + "gpt-5.6-luna", ] }; const zen: ProviderDefinition = { @@ -1012,6 +1014,7 @@ async function showThinkingEffortPicker(): Promise { { label: "Kimi (kimi-k2.*)", key: "kimi", options: ["on", "off"] }, { label: "Mimo (mimo-v2.*)", key: "mimo", options: ["off", "low", "medium", "high"] }, { label: "MiniMax (minimax-m*)", key: "minimax", options: ["off", "on"] }, + { label: "OpenAI GPT (gpt-*)", key: "openai", options: ["off", "low", "medium", "high", "xhigh"] }, { label: "Qwen (qwen3.*)", key: "qwen", options: ["auto", "on", "off"] }, { label: "Qwen Thinking Budget", key: "qwenBudget", options: ["auto", "4096", "16384", "32768", "81920"] } ]; @@ -2820,6 +2823,7 @@ function buildResponsesRequestBody( ): Record { const input = messages.flatMap((message) => responsesInputItemsFromMessage(message)); const tools = mapResponsesTools(options.tools); + const thinkingPayload = buildThinkingPayload(modelId, settings.thinking, messagesHaveImages(messages)); return { model: modelId, @@ -2828,6 +2832,7 @@ function buildResponsesRequestBody( // Only send temperature if the model supports it (not deprecated) ...(metadata.temperature !== false ? { temperature: settings.temperature } : {}), stream: true, + ...thinkingPayload, ...(tools.length ? { tools, tool_choice: toolChoice(options.toolMode) } : {}), text: { verbosity: modelId === "gpt-5-codex" ? "medium" : "low" }, }; @@ -3879,6 +3884,7 @@ function getSettings(): ApiSettings { glm: config.get("thinking.glm", "off"), kimi: config.get("thinking.kimi", "off"), minimax: config.get("thinking.minimax", "off"), + openai: config.get("thinking.openai", "off"), qwen: config.get("thinking.qwen", "off"), qwenBudget: config.get("thinking.qwenBudget", "auto"), mimo: config.get("thinking.mimo", "off"), diff --git a/src/metadata.ts b/src/metadata.ts index a000206..bae1458 100644 --- a/src/metadata.ts +++ b/src/metadata.ts @@ -174,6 +174,7 @@ const MODEL_LIMITS_BY_PROVIDER: Record updateRequestUsageSummary(usageSummary, data), + (data) => { + updateRequestUsageSummary(usageSummary, data); + rawSseData.push(data); + }, )) { + extractedPartCount += 1; reportProgressPart(localRequestId, options.progress, part); } } @@ -612,8 +620,12 @@ async function streamOpenCodeResponse( for (const part of parseServerSentEvent( buffer, options.extractStreamParts, - (data) => updateRequestUsageSummary(usageSummary, data), + (data) => { + updateRequestUsageSummary(usageSummary, data); + rawSseData.push(data); + }, )) { + extractedPartCount += 1; reportProgressPart(localRequestId, options.progress, part); } } @@ -623,6 +635,27 @@ async function streamOpenCodeResponse( `[sse-stats] totalBytes=${totalBytes} totalEvents=${totalEvents} bufferTailLen=${buffer.length}`, ); } + + // Diagnostic: when the gateway reported completion tokens but our + // extractor found nothing, dump raw SSE data to identify format mismatches. + // This helps diagnose issues like #93 where the model generates tokens + // but the response content is in an unrecognized format. + if ( + usageSummary.completionTokens + && usageSummary.completionTokens > 0 + && extractedPartCount === 0 + && rawSseData.length > 0 + ) { + options.output?.appendLine( + `[diag-empty-response] model=${options.modelId} completionTokens=${usageSummary.completionTokens} totalEvents=${totalEvents} rawSseDataCount=${rawSseData.length}`, + ); + for (let i = 0; i < rawSseData.length; i++) { + options.output?.appendLine( + `[diag-sse-event-${i}] ${truncateForLog(JSON.stringify(rawSseData[i]))}`, + ); + } + } + emitSummary(totalBytes, totalEvents, { rateLimitSummary }); } catch (error) { if (abortReason === "cancelled") { @@ -1080,7 +1113,16 @@ class OpenAiResponseExtractor { this.collectOpenAiToolCalls(message.tool_calls); } - if (first.finish_reason === "tool_calls") { + // Flush accumulated tool calls. + // Some gateways (e.g. OpenCode Go for gpt-5.6-luna) omit finish_reason + // on the final chunk even when tool calls were streamed. When + // finish_reason is null/undefined but we have pending tool calls, they + // are real and must be flushed — otherwise they silently disappear + // (issue #93). + if ( + first.finish_reason === "tool_calls" + || (first.finish_reason == null && this.pendingToolCalls.size > 0) + ) { const toolParts = this.flushToolCalls(); this.emittedToolCallsCount += toolParts.length; parts.push(...toolParts); diff --git a/src/test/thinking.test.ts b/src/test/thinking.test.ts index 0897064..6aa700f 100644 --- a/src/test/thinking.test.ts +++ b/src/test/thinking.test.ts @@ -14,6 +14,7 @@ const defaultSettings: ThinkingSettings = { glm: "off", kimi: "off", minimax: "off", + openai: "off", qwen: "off", qwenBudget: "auto", mimo: "off", diff --git a/src/thinking.ts b/src/thinking.ts index 2d5b5c1..d90d5b9 100644 --- a/src/thinking.ts +++ b/src/thinking.ts @@ -21,13 +21,14 @@ export interface ThinkingSettings { glm: "off" | "high" | "max"; kimi: "on" | "off"; minimax: "off" | "on"; + openai: "off" | "low" | "medium" | "high" | "xhigh"; qwen: "auto" | "on" | "off"; qwenBudget: "auto" | "4096" | "16384" | "32768" | "81920"; mimo: "off" | "low" | "medium" | "high"; } /** Detected thinking family for a raw model id. */ -export type ThinkingFamily = "deepseek" | "glm" | "kimi" | "minimax" | "qwen" | "mimo" | null; +export type ThinkingFamily = "deepseek" | "glm" | "kimi" | "minimax" | "openai" | "qwen" | "mimo" | null; /** * Detect which Thinking family a raw model id belongs to. Used both to render @@ -38,8 +39,7 @@ export function thinkingFamily(modelId: string): ThinkingFamily { if (/^deepseek-/i.test(modelId)) return "deepseek"; if (/^glm-/i.test(modelId)) return "glm"; if (/^kimi-/i.test(modelId)) return "kimi"; - if (/^minimax-/i.test(modelId)) return "minimax"; - if (/^qwen3(?:\.|-)/i.test(modelId)) return "qwen"; + if (/^minimax-/i.test(modelId)) return "minimax"; if (/^gpt-/i.test(modelId)) return "openai"; if (/^qwen3(?:\.|-)/i.test(modelId)) return "qwen"; if (/^mimo-/i.test(modelId)) return "mimo"; return null; } @@ -211,6 +211,28 @@ export function buildFamilyThinkingSchema( }; } + if (family === "openai") { + return { + properties: { + reasoningEffort: { + type: "string", + title: "Thinking Effort", + enum: ["off", "low", "medium", "high", "xhigh"], + enumItemLabels: ["Off", "Low", "Medium", "High", "XHigh"], + enumDescriptions: [ + "Fastest responses", + "Faster responses with less reasoning", + "Balanced reasoning and speed", + "Greater reasoning depth", + "Maximum reasoning depth" + ], + default: "off", + group: "navigation" + } + } + }; + } + if (family === "kimi") { return { properties: { @@ -359,6 +381,14 @@ export function applyRequestThinkingOverride( next.minimax = reasoningEffort as ThinkingSettings["minimax"]; } } + if (family === "openai") { + if (typeof reasoningEffort === "string") { + const valid = ["off", "low", "medium", "high", "xhigh"]; + if (valid.includes(reasoningEffort)) { + next.openai = reasoningEffort as ThinkingSettings["openai"]; + } + } + } if (family === "qwen") { if (typeof thinkingMode === "string" && (thinkingMode === "auto" || thinkingMode === "on" || thinkingMode === "off")) { next.qwen = thinkingMode; @@ -399,6 +429,17 @@ export function buildThinkingPayload(modelId: string, thinking: ThinkingSettings return { reasoning_effort: thinking.deepseek }; } + // OpenAI GPT 5.x models via Responses API: reasoning is a nested object. + // Supported values: "none", "minimal", "low", "medium", "high", "xhigh", "max". + // The OpenCode gateway forwards reasoning.effort to the OpenAI Responses API. + // Note: VS Code Copilot UI maps "max" → we map to "xhigh" for OpenAI. + if (/^gpt-/i.test(modelId)) { + if (thinking.openai === "off") { + return {}; + } + return { reasoning: { effort: thinking.openai } }; + } + if (/^glm-/i.test(modelId)) { // GLM (ZhipuAI) uses thinking: { type: "enabled" | "disabled" } format. // The gateway's transform.ts variants() returns {} for GLM — no variants From be08691b227c5a3ee7df63ff57af758ae3a907a7 Mon Sep 17 00:00:00 2001 From: ltmoerdani Date: Mon, 3 Aug 2026 02:11:09 +0700 Subject: [PATCH 2/2] docs: update CHANGELOG and issue doc for #93 fix --- CHANGELOG.md | 2 ++ docs/issues/41-20260803-gpt56-luna-routing-fix.md | 7 ++++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e0894ba..30264be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,8 @@ All notable changes to the **OpenCode Go BYOK Provider** extension are documente ### Fixed +- **`[Routing]` GPT 5.6 Luna on OpenCode Go — routing, tool calling, and reasoning (#93).** OpenCode Go docs require `gpt-5.6-luna` on the Responses API endpoint (`/v1/responses`), not chat-completions. Three issues were fixed: (1) Routing — GPT models were sent to `chat-completions` which doesn't support reasoning or tool calling for this model; now routed to Responses API per OpenCode Go docs. (2) Tool calls — the gateway sends tool calls in standard OpenAI format but omits `finish_reason` on the final chunk (`null` instead of `"tool_calls"`); `OpenAiResponseExtractor` now flushes pending tool calls when `finish_reason` is `null`/`undefined` and there are accumulated calls. (3) Reasoning — `buildResponsesRequestBody()` did not include `thinkingPayload`; added reasoning payload in Responses API format (`{ reasoning: { effort } }`). Added `openai` thinking family with effort levels (off/low/medium/high/xhigh). Diagnostic logging auto-activates when gateway reports completion tokens but extractor finds nothing. Documented in `docs/issues/41-20260803-gpt56-luna-routing-fix.md`. + - **`[Usage]` Usage monitor SVG card too narrow — values unreadable (#85).** The usage monitor SVG card (shown in status bar tooltip and webview panel) was 330px wide (345px with session data), causing the bottom statistics section to cram 6 values per line with gaps as small as 33px — making costs, request counts, and token counts hard to read. Widened the card to 420px (440px with session) and adjusted all column positions proportionally: progress bar width 256→340px, column spacing minimum 33→40px with most gaps at 80px. Tooltip image width updated to 420px, webview max-width to 560px. Documented in `docs/issues/25-20260803-usage-monitor-ui-width-fix.md`. - **`[Vision]` Trim old images from conversation history — fix `400 Upstream request failed` on MiMo + MCP screenshot loops (#38 follow-up).** Even with per-image size guards (`MAX_TOOL_RESULT_IMAGE_BYTES` 1 MB, `MAX_TOP_LEVEL_IMAGE_BYTES` 2 MB), MCP-driven agentic workflows (`chrome-devtools-mcp`, `playwright-mcp`) accumulated multiple screenshots in conversation history and hit the OpenCode Go gateway's upstream limit. Documented in `docs/issues/34-20260720-mcp-tool-result-image-dropped.md` line 264+: a `mimo-v2.5` agent loop reached `payloadBytes=4665383` (4.6 MB) after 8 screenshots and started failing every subsequent request with HTTP 400. VS Code Copilot Chat is *supposed* to trim history based on `advertisedMaxInputTokens`, but our local estimator under-counts base64 image data (`IMAGE_TOKEN_ESTIMATE = 1024` per image vs. realistic ~80K tokens/MB), so VS Code never sees the true payload weight. New function `trimOldImagesFromHistoryInPlace()` keeps only the most recent `MAX_HISTORY_IMAGES_KEPT = 2` images and replaces older ones with a short placeholder text note ("Earlier screenshot omitted from history..."). The model retains conversation structure and the latest screenshots for immediate agentic context (compare current vs. previous), while cumulative payload stays bounded. OpenAI and Anthropic vision models auto-resize each image to a 1568–2576 px patch budget upstream, so old screenshots lose most of their pixel value once a newer one arrives — the model rarely benefits from keeping more than 2 in flight. Applied after vision proxy (so proxy text descriptions are preserved) and before `promptTokens` estimation (so the output budget reflects the trimmed payload). Diagnostic log line `[history-trim] Replaced N old image(s)...` appears in the Output channel when trimming fires. diff --git a/docs/issues/41-20260803-gpt56-luna-routing-fix.md b/docs/issues/41-20260803-gpt56-luna-routing-fix.md index a9749b3..6b1f0af 100644 --- a/docs/issues/41-20260803-gpt56-luna-routing-fix.md +++ b/docs/issues/41-20260803-gpt56-luna-routing-fix.md @@ -1,7 +1,7 @@ -# Issue #41 — gpt-5.6-luna model registration + routing investigation +# Issue #41 — gpt-5.6-luna: routing, tool calling, and reasoning fix **Date:** 2026-08-03 -**Status:** ⚠️ Partial — model registered, tool calling unresolved +**Status:** ✅ Resolved **Related:** [#93](https://github.com/ltmoerdani/opencode-copilot-chat/issues/93) ## Problem @@ -77,8 +77,9 @@ Added `"gpt-5.6-luna"` to the Go provider's `fallbackModels` array so the model - ✅ Model registered in picker and metadata - ✅ Model appears in fallback list (resilient to fetch failures) -- ✅ Simple chat (no tools) works — textChars=238 +- ✅ Simple chat (no tools) works - ✅ **FIXED:** Agent mode tool calls — gateway sends tool calls but finish_reason=null, now flushed correctly +- ✅ **FIXED:** Reasoning/thinking — Responses API route + nested reasoning payload format - ✅ Diagnostic logging added for future debugging ## Diagnostic Output