Skip to content

[Measurement] Verify D1 CI gate is wired into GitHub Actions #119

Description

@AndresL230

Summary

Memory note (d1_benchmark_status.md) from 2026-05-12 claimed the D1 benchmark gate code (runner, metrics, baseline, CI workflow) was not yet built in the extension repo despite issue #86 being closed. As of PR #111 (2026-05-15), the runner / metrics / baseline / per-type gate logic clearly DO exist (we used them locally during PR-4). What's unclear is whether the gate is wired into a GitHub Actions workflow that fails CI on regression.

What to verify

  • Inspect .github/workflows/ for a job that runs npm run benchmark against benchmark/baseline.json on PR.
  • Confirm the threshold check (--threshold 1.0 per runner.ts) and the per-type gate (TP+FP >= 3 per runner.ts:265) both trip the workflow on a real regression.
  • If the workflow is missing, add one — runs on PR, blocks merge on metric drops.
  • If the workflow exists but isn't required, update branch protection.

Acceptance criteria

  • .github/workflows/ contains a benchmark job triggered on PR.
  • A test PR with an intentional metric regression (e.g., revert one of PR-4's commits) fails the CI gate.
  • Memory d1_benchmark_status.md updated or deleted depending on findings.

Reference

Memory d1_benchmark_status.md (stale).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/measurementdocs/accuracy/measurement.md — benchmark corpus and CI gatesenhancementNew feature or requestwave/5-housekeepingSmall benchmark-infra cleanups: baseline key sort, CI gate verify

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions