Skip to content

CG-020: Add fixture corpus and static benchmark harness#8

Merged
ysqander merged 4 commits intomainfrom
codex/implement-ticket-cg-020-from-docs
Mar 13, 2026
Merged

CG-020: Add fixture corpus and static benchmark harness#8
ysqander merged 4 commits intomainfrom
codex/implement-ticket-cg-020-from-docs

Conversation

@ysqander
Copy link
Copy Markdown
Owner

Motivation

  • Provide a reusable, on-disk fixture corpus and harness to validate static scanner correctness and measure performance as required by ticket CG-020.

Description

  • Replace the placeholder fixtures with a reusable fixture corpus API in @clawguard/fixtures providing listing, path resolution, deterministic SkillSnapshot loading, and content hashing via loadFixtureSnapshot and related functions.
  • Add six on-disk skill fixtures (three benign and three malicious) covering staged-download, memory-poisoning, and exfiltration scenarios under packages/fixtures/fixtures/ so tests and benchmarks can share real inputs.
  • Add fixture package tests (packages/fixtures/src/index.test.ts) and integrate scanner tests with the shared corpus expectations by using listSkillFixtures/loadFixtureSnapshot in packages/scanner/src/index.test.ts.
  • Add a static benchmark harness packages/scanner/src/benchmark-static.ts and wire bench:static scripts at the package and workspace package.json, and update docs/README entries to document the benchmark workflow.

Testing

  • pnpm install, pnpm build, and pnpm typecheck all completed successfully.
  • pnpm --filter @clawguard/fixtures test succeeded and pnpm --filter @clawguard/scanner test succeeded (scanner unit tests including the new fixture-alignment test passed).
  • pnpm bench:static executed successfully and produced benchmark JSON output for the static fixtures.
  • pnpm test at the workspace level failed due to an unrelated existing discovery test (buildSkillSnapshot returns read-failed when a file cannot be read) that appears environment-specific and is not caused by these CG-020 changes.

Codex Task

@ysqander ysqander merged commit ddf2270 into main Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant