Skip to content

contain: reduce Claude bypass prompt phrase matches - #55

Open
noogalabs wants to merge 1 commit into
mainfrom
fix/real-bypass-gate-detection
Open

noogalabs wants to merge 1 commit into
mainfrom
fix/real-bypass-gate-detection

Conversation

@noogalabs

@noogalabs noogalabs commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Temporary Containment

This PR is explicit temporary containment. It replaces broad whole-tail phrase membership with an ordered, bounded TUI feature window. The match surface is materially smaller and different from main, not a subset: it removes main's broad current-label prose trigger while admitting a smaller retitled feature-equivalent class.

The actual safety work is tracked in task_1784552757639_00544695: trusted out-of-band startup/readiness or session-phase gating.

Real Buffer Evidence

The fixture is a byte-exact 779-byte Claude Code 2.1.215 capture from a real PTY under a throwaway CLAUDE_CONFIG_DIR on 2026-07-20. Its decoded SHA-256 is 03ab7f01d0f7d44322da0f8368d7f85e2593b7439bc247de8c1c6472077bf55b.

With the production ANSI stripper:

  • running in Bypass Permissions mode: false
  • Yes, I accept: true

The detector now requires an ordered cursor-positioned header/body window, exact active selection ❯1.Noexit, and an immediately following nonempty option 2. Confirmation/cancel copy is characterization only.

Pre-existing Main Defect

This PR explicitly closes task_1784552512208_80089787: main sends keys when ordinary output quotes Bypass Permissions and Yes, I accept. That case has its own named regression.

Residual Safety Limit

Residual safety limit: This remains an output-only heuristic, not proof that the live gate is present. Any PTY output that reproduces the authorization feature pattern within the bounded window—including cursor-positioned agent, replay, or review text with the recognized header, responsibility phrase, selected ❯ 1. No exit row, and adjacent nonempty option 2—can trigger Down+Enter even when it is not byte-identical to the captured fixture and no gate is active. This change reduces incidental phrase matches; it does not provide replay immunity or ordinary-output immunity. Correctness requires a trusted out-of-band startup/readiness or session-phase signal.

This residual is evidence, not a must-trigger contract: readiness work is expected to eliminate it. The suite pins the guarantees below without preserving the limitation as required behavior.

The copy-drift asymmetry is also explicit:

  • responsibility-phrase drift produces zero keys and one warning after confirmation
  • header drift from WARNING to CAUTION produces zero keys and zero warning through 32 seconds

The header remains a silent-failure anchor. The option-2-selected rerender is protected by dual-row selection geometry, not the answer cap: either row predicate independently blocks this fixture, both weakened produce a second key, and the committed regression asserts the count remains 1.

The bounded window now uses 14 render segments. provisional three-segment slack chosen from the probe range, not measured Anthropic behaviour; +4 overflow is audible and zero-key, making a wrong bound recoverable rather than silent or destructive. By construction, this expands the feature-equivalent assembly region by three segments; the current forgery matrix found no input that exploits that growth.

The prior boundary regression inserted three segments and expected rejection because it encoded the old 11-segment bound. This PR intentionally moves that rejection control to four segments and adds a separate three-segment tolerance regression. The test change tracks the measured bound change; no assertion was removed.

Diagnostic Isolation

The unmatched warning previously wrote into the same output buffer the detector scans. A real gate split around the warning could therefore lose its option row from the bounded window and remain unanswered. The detector now excludes the exact internal warning segment while retaining it for operators. A red-first prefix -> warning -> suffix regression proves the diagnostic no longer changes classifier input.

This exact-string exclusion is bounded to the current internal writer. Structural separation of internal diagnostics from detector input is tracked in task_1784554889660_60163313.

Executable Evidence

Defect and review cases:

  • current-label review text: main and the first pushed head wrote a key
  • retitle fallback prose in one chunk: first pushed head wrote a key
  • fallback tokens split across chunks: first pushed head wrote a key
  • warning prose: first amendment emitted a false warning
  • responsibility collapsed into the header: replacement predecessor wrote a key
  • delayed real suffix after the internal warning: replacement predecessor wrote zero keys
  • feature-equivalent cursor-positioned output remains a disclosed triggering class

Positive and containment controls:

  • real fixture: exactly one Down+Enter
  • retitled option with rewritten confirmation chrome: exactly one Down+Enter
  • partial TUI-backed gate: one warning after confirmation, zero keys
  • responsibility drift: zero keys, exactly one warning
  • header drift to CAUTION: zero keys and zero warnings through 32 seconds
  • named A/B/C and warning-prose cases: zero keys and zero warnings
  • missing active selection, reordered rows, out-of-window rows, stripped output, and cursor-positioned out-of-order prose: zero keys
  • empty adjacent option 2: zero keys, exactly one warning
  • option 2 already selected: no second key, one warning, answer count remains 1
  • three inserted segments: exactly one Down+Enter, zero warnings
  • four inserted segments: zero keys, exactly one warning
  • real prefix, internal warning, then real suffix: exactly one Down+Enter
  • four near-match ordered-tui-v2 diagnostics remain classifier input and prevent stitching
  • warning path does not call heartbeat-minting logEvent
  • warning re-arms only on a fresh PTY lifecycle

Load-bearing mutations:

  • accepting unselected option 1 fails the selection regression
  • dropping cursor-position evidence fails the stripped-output regression
  • searching option 2 anywhere fails the adjacency regression

Green bar on the final local artifact:

  • focused: 58/58
  • serialized full: 197 files, 3,639 tests
  • clean-build parallel full: 197 files, 3,639 tests
  • typecheck, isolated build, diff check, and real-config canary passed

The first parallel rerun after the ordering delta hit the repository's known tsup/dist race when another test removed dist/cli.js; a clean-build rerun passed. During the production delta, an unsandboxed serialized run passed all 3,634 tests but correctly exited nonzero when an unrelated live Claude process rewrote the host-global config. The authoritative serialized and parallel runs for this test-only amendment used a disposable outer HOME; both passed all 3,639 tests and the canary.

The unmatched warning uses console.error and the PTY output buffer only. It deliberately avoids logEvent, whose current side effect refreshes heartbeat.

No live checkout was built or tested.

@noogalabs
noogalabs force-pushed the fix/real-bypass-gate-detection branch 2 times, most recently from 6c7f1b7 to b3a8407 Compare July 20, 2026 13:25
@noogalabs noogalabs changed the title fix: harden Claude bypass prompt detection contain: reduce Claude bypass prompt phrase matches Jul 20, 2026
@noogalabs

Copy link
Copy Markdown
Owner Author

@codex review exact commit b3a8407

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3a84074fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/pty/agent-pty.ts
' runtime=claude-code detector=ordered-tui-v1' +
' action=manual-intervention-required';
console.error(warning);
this.outputBuffer.push(`${warning}\n`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the warning out of the prompt stream

When the real Claude bypass dialog arrives in chunks and the URL/options portion is delayed until after the second retry, this synthetic line is inserted between the already-buffered header/responsibility lines and the later option rows. Because classifyBypassGate() only scans an 11-segment window from the header, the inserted warning shifts ❯ 1. Noexit/2. ... out of that window, so a gate that becomes fully actionable after the warning remains classified as unmatched and the unattended agent never sends Down+Enter.

Useful? React with 👍 / 👎.

@noogalabs
noogalabs force-pushed the fix/real-bypass-gate-detection branch from b3a8407 to 387eed4 Compare July 20, 2026 13:57
@noogalabs
noogalabs force-pushed the fix/real-bypass-gate-detection branch from 387eed4 to fa18ebf Compare July 20, 2026 14:12
@noogalabs

Copy link
Copy Markdown
Owner Author

@codex review exact commit fa18ebf

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: fa18ebf94b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant