Skip to content

fix(agent,shared): an empty model reply finished the turn as a success, and Stop was reported as a crash - #196

Merged
sebyx07 merged 2 commits into
mainfrom
fix/no-output-and-abort-noise
Aug 26, 2026
Merged

sebyx07 merged 2 commits into
mainfrom
fix/no-output-and-abort-noise

Conversation

@sebyx07

@sebyx07 sebyx07 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

What

Two production crash classes from GlitchTip project ai-designer (5 unresolved issues, 21 events, 2026-07-31 to 2026-08-18):

  • AI_NoOutputGeneratedError — 19 events across issues 67 (2x), 75 (14x), 91 (3x), 93 (1x). A turn that produces nothing now says so, loudly and by name, instead of finishing as a silent success.
  • AbortError — 2 events, issue 76. The user pressing Stop is no longer reported to GlitchTip as a crash.

Plus: every report ships a release now, so the next one can be tied to a build.

Why

Where the error is made. Both classes are the same one line of ai@7.0.19: the ternary in streamText's eventProcessor flush, which hands rejectResultPromises either abortSignal.reason (an AbortError) or a fresh NoOutputGeneratedError. Confirmed by column, not by guess: building this repo at ab2527e (the tip while issue 75 was firing) puts new NoOutputGeneratedError({message: "No output generated. Check the stream for errors."}) at background.js line 115 column 36989 — the exact frame all 14 events of issue 75 carry. Issue 76's AbortError frames land in the message handler beside the turnAbort?.abort() calls.

AbortError is not a crash. Nothing here aborts on a deadline. All five AbortController.abort() calls live in background.ts and all five are a deliberate end-of-turn: supersede, session-start, session-stop (the Stop button), conversation-new, and the cross-document nav abort in onCommitted. AbortSignal.timeout appears nowhere in src/, and the only other abortable waits (waitForTabComplete, browseDelay) reject with a plain Error. Each of the five already ends the turn with a reason the user sees — runTurn returns stop: 'aborted', and the nav abort posts its own message. Reporting it as an unhandled crash is noise, and noise is what makes the real reports get skimmed. It is dropped at the Sentry seam only: background.ts's unhandledrejection listener still writes it to the conversation's debug log, and the worker console still prints it.

AI_NoOutputGeneratedError had a silent twin, and the twin is the user-visible bug. The SDK raises that error only when the stream carried no completed step at all — a 401/404/5xx on the first model call, which runTurn already catches and shows. But an OpenAI-compatible gateway that answers 200 and then closes still yields a finish chunk, so a step IS recorded, streamText reports a clean empty finish, and runTurn returned stop: 'done' with an empty reply, no error, and nothing on screen. Driving the real @ai-sdk/openai-compatible parser against a stub fetch, three realistic gateway replies all land there:

gateway reply (HTTP 200) before after
empty text/event-stream body stop: done, empty, silent stop: error + named message
JSON error object served as an event stream stop: done, empty, silent stop: error + named message
chunk carrying choices: [] stop: done, empty, silent stop: error + named message

That is "the model returned no output" as the user meets it, and all three are how a gateway says that model id resolved to no provider without an HTTP status the SDK's retry would act on. The config is the first thing to check, so the message says so.

Changes

  • src/agent/loop.ts — track whether the turn asked for any tool; a natural finish (stop === 'done') with no prose and no tool call flips to stop: 'error' and emits EMPTY_TURN_ERROR. Gated on done so it can only describe a turn the model ended itself: aborted, budget and error each already carry their own reason and never reach it.
  • src/shared/sentry.tsisUserAbort (exported, enumerated in its own doc comment); beforeSend drops those events and scrubs the rest. scrubEvent is untouched: privacy scrub and worth-reporting stay two seams. release is now set to designer@<manifest version> — the field scrubEvent has always allowlisted and nothing ever populated. integrations now subtracts the SDK's BrowserSession default — see The release stamp had a side effect below.
  • test/integration/empty-turn.test.ts (new) — the three gateway replies plus a bare mock finish, each pinned to EMPTY_TURN_ERROR by identity, not substring; two anti-vacuity partitions (a tools-only turn and a gateway that streams real content) must stay done.
  • test/unit/sentry.test.ts — the abort drop asserted through the wired beforeSend, AI_NoOutputGeneratedError asserted to still ship (scrubbed), and isUserAbort false for a mixed multi-exception event and for an exception-less one.
  • test/unit/sentry.test.ts — plus the session-integration guard: the wired reducer is run over the real getDefaultIntegrations() set and must drop BrowserSession and keep every other entry. It asserts the default set contains BrowserSession first, so it cannot pass by filtering a name the SDK no longer ships.

The release stamp had a side effect

Stamping release armed a lane nobody asked for. browserSessionIntegration is in the browser SDK's default integration set and calls captureSession() from its setupOnce; it had been inert here purely by accident, because Client.sendSession discards a session when the client has no release. So the moment this branch set one, every side panel mount POSTed a session envelope to GlitchTip before anything had gone wrong.

Found by CI, not by reading: test/e2e/smoke.spec.ts ("side panel makes zero blocked/remote font or script requests") went red on a glitchtip.infra.developerz.ai/api/2/envelope/ request from a freshly opened panel, and reproduces locally against the built extension.

It is removed rather than tolerated, for two reasons. It is unsolicited network traffic from a panel at rest, in an extension whose entire crash-report seam exists to keep things inside the browser. And beforeSend cannot police it: that hook sees error events, not session envelopes, so both the scrub and the isUserAbort drop sit downstream of a lane they never see.

The filter is subtractive — keep whatever the installed SDK ships by default, minus the one lane refused — and the list's element type is derived from Sentry.getDefaultIntegrations rather than re-declared, because @sentry/browser does not re-export Integration.

Verification

  • Observed red first. Reverting the loop.ts guard fails 4 of the 6 new integration tests and leaves both anti-vacuity partitions green; reverting the sentry.ts wiring fails exactly the 2 new initSentry tests.
  • The stop === 'done' gate is what keeps the two states apart, and that is pinned by an existing test, not by reasoning: drop the gate and agent-loop.test.ts's "reports an aborted turn without emitting an error" goes red with expected 'error' to be 'aborted' — a Stop would be re-reported as an empty turn. Run red, then restored.
  • The session-integration guard was run red the same way: without integrations, the new unit test fails with "Sentry.init was not called with an integrations reducer".
  • bun run gate clean — lint (415 files), typecheck (tsgo), 2709 unit + integration tests in 197 files, build, and 56 Playwright e2e tests against the loaded extension, including the smoke test that caught the envelope.

Follow-up

Not fixed, and stated plainly: why these errors reach unhandledrejection at all is still open. runTurn already parks a catch on result.steps / finishReason / totalUsage, and ai@7.0.19 additionally calls markPromiseAsHandled on all five promises rejectResultPromises touches — verified present in the minified ab2527e bundle that produced the reports. Eleven scenarios reproduced against both MockLanguageModelV4 and the real provider (empty stream, no finish chunk, provider error part, transport error mid-stream, HTTP 404, retries exhausted, abort mid-stream, step-2 empty, tool throw, and the three gateway replies above) produce zero unhandled rejections on current main. So the residual leak path is unidentified. The release stamp is what closes that loop: the next report will name its build, which is precisely what issue 93 (a single event on 2026-08-18, from an install that could not be dated) could not.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…s, and Stop was reported as a crash

Two crash classes from GlitchTip project `ai-designer` (19 events of
`AI_NoOutputGeneratedError` across issues 67/75/91/93, 2 of `AbortError`
in issue 76). Both are made at the same line of `ai@7.0.19` — the ternary
in `streamText`'s `eventProcessor` flush that hands `rejectResultPromises`
either `abortSignal.reason` or a fresh `NoOutputGeneratedError`. Confirmed
by column: building this repo at `ab2527e` puts that `new` at
`background.js` 115:36989, the exact frame all 14 events of issue 75 carry.

The AbortError is the user pressing Stop. All five `AbortController.abort()`
calls live in `background.ts` and all five are a deliberate end-of-turn
(supersede, session-start, session-stop, conversation-new, nav abort);
`AbortSignal.timeout` appears nowhere in `src/`, and the other abortable
waits reject with a plain `Error`. Each already ends the turn with a reason
the user sees, so reporting it as a crash is noise. Dropped at the Sentry
seam only — `background.ts`'s `unhandledrejection` listener still logs it.

The no-output error had a SILENT twin, and the twin is the user-visible
bug. The SDK raises it only when no step completed at all; a gateway that
answers 200 and closes still yields a finish chunk, so a step is recorded
and the turn returned `stop: 'done'` with an empty reply and no error. An
empty event-stream body, a JSON error object served as an event stream, and
a chunk with `choices: []` all landed there — all three are how an
OpenAI-compatible gateway says "that model id resolved to no provider"
without a status the SDK's retry would act on. Now named.

Also stamps `release` (`designer@<manifest version>`) — the field
`scrubEvent` has always allowlisted and nothing ever set, which is why
issue 93 could not be tied to a build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

On-demand reviews are free for the next 25 days. After that, they cost $0.25 per reviewed file.

Or wait 33 minutes for your next included review.

View limit details

Limit details: You’ve used the included review currently available. Your 77 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e2234d23-673f-4861-9ff4-845121b6943b

📥 Commits

Reviewing files that changed from the base of the PR and between 37394ac and d2f4551.

📒 Files selected for processing (4)
  • src/agent/loop.ts
  • src/shared/sentry.ts
  • test/integration/empty-turn.test.ts
  • test/unit/sentry.test.ts

Warning

.coderabbit.yaml has a parsing error

The CodeRabbit configuration file in this repository has a parsing error and default settings were used instead. Please fix the error(s) in the configuration file. You can initialize chat with CodeRabbit to get help with the configuration file.

Parsing errors (1)
Validation error: Too big: expected string to have <=250 characters at "tone_instructions"
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Comment @coderabbitai help to get the list of available commands.

@blacksmith-sh

This comment has been minimized.

…e panel phoned home at rest

`browserSessionIntegration` is in the browser SDK's DEFAULT set and calls
`captureSession()` from its `setupOnce`. It had been inert here only by
accident: `Client.sendSession` discards a session when the client has no
`release`. Stamping `release` in this branch is exactly what armed it, so every
side panel mount now POSTed a session envelope to GlitchTip before anything had
gone wrong.

Caught by `test/e2e/smoke.spec.ts` ("side panel makes zero blocked/remote font
or script requests"), which went red on CI and reproduces locally against the
built extension.

It comes out rather than being tolerated. It is unsolicited network traffic from
a panel at rest, in an extension whose whole crash-report seam exists to keep
things inside the browser; and `beforeSend` cannot police it, because that hook
sees error events, not session envelopes — both the scrub and the `isUserAbort`
drop sit downstream of a lane they never see.

The filter is subtractive (keep the SDK's defaults, minus the one lane we
refuse) and the integration list type is derived from
`Sentry.getDefaultIntegrations` rather than re-declared, since `@sentry/browser`
does not re-export `Integration`.

Guard: a unit test asserts the wired reducer removes `BrowserSession` from the
REAL default set and keeps every other entry — non-vacuous, because it first
asserts the default set actually contains it. Observed red before the fix
("Sentry.init was not called with an integrations reducer"). The stale
`not.toHaveProperty('integrations')` assertion the fix invalidates is replaced
by that test; the replay/tracing assertions beside it are untouched.
@sebyx07

sebyx07 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@sebyx07
sebyx07 merged commit ae7ccd6 into main Aug 26, 2026
7 checks passed
@sebyx07
sebyx07 deleted the fix/no-output-and-abort-noise branch August 26, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant