Skip to content

fix(#6821): add Go-side context deadline and JS-side recovery for WASM mint timeout - #6823

Open
fullsend-ai-coder[bot] wants to merge 2 commits into
mainfrom
agent/6821-mint-wasm-timeout-recovery
Open

fix(#6821): add Go-side context deadline and JS-side recovery for WASM mint timeout#6823
fullsend-ai-coder[bot] wants to merge 2 commits into
mainfrom
agent/6821-mint-wasm-timeout-recovery

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Prevent permanent mint WASM isolate poisoning after GitHub API timeouts by adding Go-side context deadlines and replacing the JS-side permanent-poison mechanism with automatic recovery.

Related Issue

Closes #6821

Changes

Go-side (primary fix):

  • Add awaitPromiseWithContext in internal/mintcore/fetch_js.go — a context-aware variant of awaitPromise that returns ctx.Err() on cancellation while a background goroutine cleans up JS callback resources
  • Update mintHTTP in http_client_js.go to use the context-aware variant, honoring request context deadlines on WASM
  • Update HostPEMAccessor.AccessPEM in pem_js.go to honor the context parameter (previously ignored)
  • Add 20s per-request context deadline in cmd/mint-wasm/main.go (requestTimeout), 5s below the JS-side 25s HANDLE_FETCH_TIMEOUT_MS, so slow GitHub API calls return clean Go errors before the JS timeout fires

JS-side (defense-in-depth):

  • Replace permanent poisoning (markPoisoned) with recovery (markTimedOut) in GoWasm class
  • After timeout, clear initPromise so the next request boots a fresh Go WASM runtime instead of returning 503
  • Old Go runtime leaks (bounded by CF isolate lifetime) but cannot interfere — its globalThis exports are overwritten by the new runtime

Tests:

  • Add TestFindInstallation_ContextDeadline and TestFindOrgInstallation_ContextDeadline verifying context deadline behavior on native platform

Embed sync:

  • Updated .embed copies for fetch_js.go, http_client_js.go, pem_js.go

Testing

  • go test -race -count=1 ./... passes in internal/mintcore/
  • go vet ./... passes
  • TestEmbeddedMintSource passes (embed sync verified)
  • lint-mint-embed-sync passes
  • Secret scan passes
  • New context deadline tests pass

Closes #6821

Post-script verification

  • Branch is not main/master (agent/6821-mint-wasm-timeout-recovery)
  • Secret scan passed (gitleaks — 1ac1750a661c6ccc170267c8b3919d2597cf7810..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

…M mint timeout

The mint WASM isolate permanently poisons after a GitHub API timeout,
causing cascading 503s for all subsequent callers until isolate recycle.

Root cause: the WASM-side mintHTTP did not honor request context
deadlines — awaitPromise blocked until the JS Promise settled regardless
of Go context cancellation. When a GitHub API call exceeded the JS-side
25s timeout (HANDLE_FETCH_TIMEOUT_MS), the GoWasm singleton was
permanently poisoned with no programmatic recovery.

Go-side fix (primary):
- Add awaitPromiseWithContext in fetch_js.go that respects context
  cancellation, returning ctx.Err() immediately while a background
  goroutine cleans up the JS callback resources after the Promise settles
- Update mintHTTP (http_client_js.go) and HostPEMAccessor.AccessPEM
  (pem_js.go) to use the context-aware variant
- Add a 20s per-request context deadline in cmd/mint-wasm/main.go
  (requestTimeout), 5s below the JS-side 25s timeout, so slow GitHub
  API calls surface as clean Go-side errors before the JS timeout fires

JS-side fix (defense-in-depth):
- Replace permanent poisoning with recovery: after a timeout, the GoWasm
  singleton is marked for recovery (markTimedOut) instead of permanently
  poisoned (markPoisoned). The next request re-initializes the Go WASM
  runtime via doInit rather than returning 503
- The old Go runtime leaks (bounded by CF isolate lifetime) but cannot
  interfere: its globalThis exports are overwritten by the new runtime,
  and late Promise resolutions are silently ignored

Closes #6821
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 31, 2026 20:18
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Agent PR ready for human review label Aug 31, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 31, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:20 PM UTC · Completed 8:40 PM UTC

Commit: afc3796 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $6.08

@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Aug 31, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 31, 2026

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Targeted bug fix for a high-severity issue (permanent WASM isolate poisoning) with clear scope across 9 files (385 lines). Low Tier 1 signals (no protected paths, no dependency changes, bot author), elevated Tier 2 (high churn and regression history in changed mint files), moderate Tier 3 (well-scoped issue with clear acceptance criteria). Score unchanged from prior assessment.

Previous run

Risk Assessment: moderate (2/5)

Details

Targeted bug fix for a high-severity issue (permanent WASM isolate poisoning) with clear scope across 9 files (331 lines). Low Tier 1 signals (no protected paths, no dependency changes, bot author), elevated Tier 2 (high churn and regression history in changed mint files), moderate Tier 3 (well-scoped issue with clear acceptance criteria).

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 31, 2026

Copy link
Copy Markdown

Looks good to me

Previous run

Review

Findings

High

  • [test-integrity] internal/mintcore/github_test.go:658 — The ErrorIs assertion in TestFindInstallation_ContextDeadline is a tautology: assert.ErrorIs(t, context.DeadlineExceeded, context.DeadlineExceeded) compares context.DeadlineExceeded against itself, which always passes regardless of the actual error. The returned err is never type-checked, so the test would pass even if FindInstallation returned an unrelated error — defeating the stated purpose of verifying context deadline behavior.
    Remediation: Change to assert.ErrorIs(t, err, context.DeadlineExceeded).

Medium

  • [test-adequacy] internal/mintcore/github_test.go:677TestFindOrgInstallation_ContextDeadline only asserts require.Error(t, err) without verifying the error wraps context.DeadlineExceeded. The test would pass for any error type, not just context deadline errors.
    Remediation: Add assert.ErrorIs(t, err, context.DeadlineExceeded).

  • [comment-staleness] internal/dispatch/cf/workersrc/src/index.ts:516 — The goWasm singleton comment states "the singleton is never replaced," which is misleading now that the instance re-initializes a fresh Go WASM runtime internally after timeout. The const reference is never reassigned, but the Go runtime underneath is replaced.
    Remediation: Clarify that the object reference is const but the instance boots a fresh Go runtime internally after timeout recovery.

Low

  • [availability] internal/dispatch/cf/workersrc/src/index.ts:553 — Each timeout-recovery cycle leaks one Go WASM runtime (blocked goroutine + memory). Under sustained upstream slowness, multiple runtimes could accumulate before Cloudflare evicts the isolate. The Go-side 20s context deadline substantially reduces the probability of reaching the JS-side timeout, and Cloudflare's isolate memory limits provide a backstop.
    Remediation: Consider adding a recovery attempt counter; after N consecutive timeouts (e.g., 3), revert to permanent 503 to bound leaked runtimes.

  • [comment-staleness] cmd/mint-wasm/main.go:155 — The comment "permanently poisons the GoWasm singleton" uses terminology from the old behavior that this PR replaces with a recovery model.
    Remediation: Change "permanently poisons" to "blocks" or "stalls" to align with the new recovery semantics.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 8:42 PM UTC · Completed 8:50 PM UTC

Commit: afc3796 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.18

- Fix tautological ErrorIs assertion in TestFindInstallation_ContextDeadline
  (was comparing context.DeadlineExceeded against itself instead of err)
- Add ErrorIs assertion to TestFindOrgInstallation_ContextDeadline to
  verify the error wraps context.DeadlineExceeded
- Add consecutive timeout recovery counter to GoWasm class (cap at 3)
  to bound leaked WASM runtimes under sustained upstream slowness
- Update stale comments: "permanently poisons" → recovery-aware wording,
  clarify const singleton comment re: internal runtime replacement

Addresses #6823
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 1 (bot-triggered)

Addressed all 5 review findings: fixed tautological ErrorIs assertion, added missing ErrorIs check in org installation test, added recovery counter to bound leaked WASM runtimes (cap at 3 consecutive timeouts), and updated two stale comments to match recovery semantics.

Fixed (5):

  1. The ErrorIs assertion in TestFindInstallation_ContextDeadline is a tautology (internal/mintcore/github_test.go): Changed assert.ErrorIs(t, context.DeadlineExceeded, context.DeadlineExceeded) to assert.ErrorIs(t, err, context.DeadlineExceeded) so the test actually verifies the returned error wraps context.DeadlineExceeded
  2. TestFindOrgInstallation_ContextDeadline only asserts require.Error without verifying the error wraps context.DeadlineExceeded (internal/mintcore/github_test.go): Added assert.ErrorIs(t, err, context.DeadlineExceeded) after require.Error to verify the error type, not just its presence
  3. The goWasm singleton comment states the singleton is never replaced, which is misleading now that the instance re-initializes internally (internal/dispatch/cf/workersrc/src/index.ts): Clarified that the const object reference is never reassigned but the instance internally boots a fresh Go WASM runtime after timeout recovery
  4. Each timeout-recovery cycle leaks one Go WASM runtime; under sustained upstream slowness multiple runtimes could accumulate (internal/dispatch/cf/workersrc/src/index.ts): Added a consecutive timeout counter (_consecutiveTimeouts) and MAX_CONSECUTIVE_RECOVERIES cap (3). After 3 consecutive timeouts the instance returns 503 until isolate recycle. Counter resets on successful request completion.
  5. The comment permanently poisons the GoWasm singleton uses terminology from the old behavior (cmd/mint-wasm/main.go): Changed 'permanently poisons the GoWasm singleton' to 'blocks the GoWasm singleton until the JS-side timeout triggers recovery' to align with the new recovery semantics

Tests: passed

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 31, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:52 PM UTC · Completed 9:07 PM UTC

Commit: 83de70f · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.17

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review August 31, 2026 21:07

Superseded by updated review

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge ready-for-review Agent PR ready for human review risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mint WASM isolate poisons permanently after GitHub API timeout, causing cascading 503s

1 participant