Skip to content

Bump corpus SHA + baseline for A2/A7 fixtures - #105

Merged
AndresL230 merged 1 commit into
mainfrom
claude/corpus-bump-a2-a7
May 13, 2026
Merged

Bump corpus SHA + baseline for A2/A7 fixtures#105
AndresL230 merged 1 commit into
mainfrom
claude/corpus-bump-a2-a7

Conversation

@AndresL230

Copy link
Copy Markdown
Contributor

Summary

Bumps .benchmark-fixtures-sha to 217ab45 (corpus PR recost-dev/extension-benchmark#6 merged) so the D1 gate now scans 7 fixtures including:

Updates benchmark/baseline.json to reflect the new aggregate metrics — A2 and A7 are now measured.

D1 baseline change

Metric Before After Δ
Detection precision 30.95% 36.26% +5.31pp
Detection recall 42.62% 48.53% +5.91pp
Provider attribution 79.59% 82.14% +2.55pp
Finding precision 7.14% 6.25% -0.89pp
Finding recall 33.33% 33.33% +0.00pp

Finding precision dips slightly (-0.89pp, well within the 1pp gate tolerance): the two new fixtures have no expected findings, but raw-fetch-elevenlabs emits at least one finding (likely a local-rule waste suggestion on the new raw fetch calls) that counts as FP in the aggregate denominator. Detection metrics are the load-bearing signal here, and they jump as predicted by the dry-run (+5.31pp / +5.91pp / +2.55pp).

Both new fixtures hit 100/100 detection P/R on the live scanner — proving A2 and A7 work in measurement, not just in in-repo tests:

[dynamic-fetch-urls]    det P/R 100.0/100.0 | TP/FP/FN endpoints 4/0/0
[raw-fetch-elevenlabs]  det P/R 100.0/100.0 | TP/FP/FN endpoints 3/0/0

No existing fixture regressed.

Test plan

  • CI benchmark workflow passes with the new SHA + baseline

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@AndresL230 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 13 minutes and 13 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c9c261b6-4bed-4b33-9cf8-8c446f4c0a1c

📥 Commits

Reviewing files that changed from the base of the PR and between 1b0d029 and 7bd717e.

📒 Files selected for processing (2)
  • .benchmark-fixtures-sha
  • benchmark/baseline.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/corpus-bump-a2-a7

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@AndresL230
AndresL230 merged commit 79bc3cf into main May 13, 2026
3 checks passed
@AndresL230
AndresL230 deleted the claude/corpus-bump-a2-a7 branch May 22, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant