fix(pairing): add call-site verification to strategy-1 name matching (#14) - #16
Conversation
…14) - Add optional test_tree parameter to _name_match for call-site verification - Verify name-convention matches against direct calls in test body via _direct_call_names before emitting pairings - Suppress false pairings where test name matches target but never calls it - Update 14 existing tests to include target function calls in test bodies - Add 8 new tests: direct/attribute/suppression/fallthrough/indirect/ case-only/integration coverage - Add OpenSpec change artifacts (proposal, design, spec, tasks) Closes #14 Assisted-by: claude-opus-4 Generated with AI assistance (claude-opus-4)
jflowers
left a comment
There was a problem hiding this comment.
Council Verdict: APPROVE
Note: Could not post as APPROVE due to GitHub's self-review prohibition (author cannot approve own PR). Posted as COMMENT instead. Original verdict: APPROVE.
Reviewers: Adversary, Architect, Curator, Envoy, Guard, Herald, Scribe, SRE, Testing
Iterations: 1
Adversary (APPROVE)
No findings.
Architect (APPROVE)
2 LOW findings omitted.
Curator (APPROVE)
No findings.
Envoy (APPROVE)
3 LOW findings omitted.
Guard (APPROVE)
1 LOW finding omitted.
Herald (APPROVE)
3 LOW findings omitted.
Scribe (APPROVE)
5 LOW findings omitted.
SRE (APPROVE)
1 LOW finding omitted.
Testing (APPROVE)
3 LOW findings omitted.
This review was generated by /review-council (AI-assisted).
PR Cost Report: #16Session: Change proposal artifact workflowID:
Timeline: 2026-09-01 17:25:40 — 2026-09-02 08:33:07 Child sessions: 16 — additional cost: $15.58 Child session breakdown
Session: Code and spec review councilID:
Timeline: 2026-09-02 08:33:16 — 2026-09-02 09:27:47 Child sessions: 10 — additional cost: $12.23 Child session breakdown
Grand Total: $43.04 across 2 session tree(s) (28 sessions) |
Summary
Strategy-1 (name-convention) pairing in
quality/pairing.pynow verifiescall-site presence before emitting a pairing. When a test function's name
matches a target via prefix stripping (e.g.,
test_add→add), thesystem walks the test function's AST body for direct calls to the matched
target. If no matching call is found, the pairing is suppressed entirely,
allowing the test to fall through to strategy-2 or strategy-3.
This eliminates false pairings where a test is named after a function it
never actually invokes, improving Gaze's test quality metrics accuracy.
Closes #14
How to Test
How to Demo
def add(): ...and a test file containing
def test_add(): assert True(no call toadd)snake-eyes --stdioand send atest_mappingrequesttest_adddoes NOT pair withaddat confidence 90(previously it would have been a false positive)
add()to the test body and re-run — now the pairing is emitted correctlyKey Files Changed
src/snake_eyes/quality/pairing.py— Added optionaltest_treeparameterto
_name_match; call-site verification via_direct_call_namesbeforeemitting pairings (both exact-match and case-only paths)
tests/test_test_mapping_method.py— Updated 14 existing tests to includetarget calls; added 8 new tests covering call-site verification scenarios
openspec/changes/fix-strategy1-callsite-verification/— OpenSpec changeartifacts (proposal, design, spec with 8 scenarios, tasks with 15 items)
This PR was generated by /uf.finale (AI-assisted).