feat(bench): 3 TypeScript cascade scenarios (#104)#125
Merged
Connorrmcd6 merged 1 commit intoJun 14, 2026
Merged
Conversation
PR-B of the cascade expansion: three TS code cascades (graded by `node --test`, Node >=22.18 type-strip), each a new drift archetype that surf detects via a structural/literal change (not an identifier swap). - cascade-serialize-key-order-ts-code (T2, serialization) — canonicalString key order insertion -> sorted (Object.keys().sort() added); the visible signer must reproduce the canonical string. - cascade-default-timeout-ts-code (T1, timeouts) — totalDeadlineMs default param 30000 -> 5000; the visible caller reports the default budget. - cascade-validate-guard-ts-code (T2, validation) — isValidName dropped its `length > 0` guard (empty names now accepted); the visible intake filter must mirror the rule. Each: graders probe the real hidden dependency for ground truth, task.md is neutral, author.py seals a genuine "changed" divergence, and validate_scenario proves grader polarity. Verified offline: validate_scenario --all green (7/7), a mock run over the three is error-free, uv run pytest passes (38). Stacked on PR-A (#124) for the validate_scenario tooling + edit_path; will retarget to main once #124 merges. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR-B of the cascade-family expansion: 3 TypeScript code cascades (graded by
node --test, Node ≥22.18 type-strip). Each introduces a new drift archetype thatsurfdetects via a structural/literal change (not an identifier swap — see PR-A's note).New scenarios
cascade-serialize-key-order-ts-code.sort()added)cascade-default-timeout-ts-code30000→5000cascade-validate-guard-ts-codelength > 0guard (empty now accepted)Each: graders probe the real hidden dependency for ground truth,
task.mdis neutral (no leaked value),author.pyseals a genuine"changed"divergence, andvalidate_scenarioproves the graders discriminate.Verification (offline)
python tools/validate_scenario.py --all— 7/7 green (4 from PR-A + these 3).uv run pytest— 38 passed;cargo fmt --all --checkclean.Scope
PR-B of Connorrmcd6/surface-bench#12. Cascade family now 4 → 11 (with PR-A). PR-C (3 QA cascades) reaches ~14 and unblocks the phase-8 pre-registration freeze.
🤖 Generated with Claude Code