Epic: Productionize scheduled benchmark operations

[amirbena]

Design provenance

Design provenance: #464, PR #465. #464 established the production
operations architecture for scheduled benchmark execution (decision
record under runtime_platform/benchmark/scheduled-operations/).
This Epic implements that architecture; it does not reopen #464's
research, and #464 is not turned into an implementation parent.

What this Epic delivers

• Benchmark execution stays outside GitHub Actions. Execution (scheduling,
  running, in-run drift confirmation) produces a sealed, immutable
  canonical result (benchmark-result/v1) before any GitHub write is
  attempted.
• Publication is a deterministic, publication-only GitHub Actions
  workflow — schedule sweep plus manual workflow_dispatch only, a
  default-branch workflow definition, no model or provider credential, no
  drift derivation, and no benchmark imports. It never judges review
  output; it executes a publication plan already decided by the sealed
  record.
• Writes to GitHub use short-lived installation tokens minted inside the
  job from a dedicated benchmark-publication GitHub App (phase-scoped:
  contents: write then issues: write), distinct from the release App.
  The workflow's own GITHUB_TOKEN stays contents: read.
• Persistent records — the sealed result, the publisher's receipt, and a
  per-lane baseline pointer — live on the benchmark-history branch under
  create-only, hash-checked, content-addressed contracts.
• Drift → issue lifecycle is deterministic and lane-aware: an open
  issue closes only once every scheduled lane that currently covers that
  case stops reproducing it, so a sentinel result can never close a
  comprehensive-only regression (amendment A9).
• The same publication-only workflow's schedule trigger also runs a
  missed-run watchdog — gap-based, metadata-only, never executing or
  evaluating the benchmark.
• Publication failure never causes benchmark re-execution: retry always
  operates from the sealed handoff, never from a fresh run.

Implementation dependency graph

approve amendments A1–A13 (recorded on #467)
        │
        ▼
       #467  F1 — Land approved amendments (docs-only)
        │
        ├──────────────┬───────────────┐
        ▼               ▼               ▼
       #468            #469            #473
  F2 — result schema  F3 — schedule   F7 — provisioning spec:
  + validator          spec, manifest, App, environment,
  (gated A4/A7)         labels          rulesets, labels
        │               │               │
        ├───────┬───────┤               │
        ▼       ▼       ▼               │
       #470    #471    #476             │
   F4 —      F5 —     F10 —             │
   execution  publish  Actions          │
   entrypoint CLI:     read-only        │
   (gated     persist/ validation       │
   A2/A8)     reconcile                 │
              (gated                    │
              A1/A8/A9)                 │
        │       │                       │
        │       ▼                       │
        │      #472                     │
        │   F6 — watchdog +             │
        │      health status            │
        │       │                       ▼
        │       │                      #484
        │       │              F7 — execution:
        │       │              provision + verify
        │       │                       │
        │       └───────────┬───────────┘
        │                   ▼
        │                  #474  F8 — Publication workflow (schedule + workflow_dispatch,
        │                       gated A13 — "approval of A13 is what allows F8")
        │                   │
        │                   ▼
        └─────────────────▶#475  F9 — Routine provisioning, verification, first verified runs
                            │
                            ▼
                           #477  F11 — Repository operations specification (canonical docs)
                            │
                            ▼
                           #478  F12 — GitHub Wiki operating-model page

Execution order groups: (1) #467 establishes data/contracts; (2) #470
fixes existing benchmark execution semantics; (3) #471/#472 build
publication; (4) #473 specifies and #484 executes privileged GitHub
provisioning (parallel with code); (5) #474 deploys the scheduled
publication workflow; (6) #475 proves end-to-end operation; (7) #472 adds health/watchdog behavior
(parallel with #471); (8) #477/#478 complete canonical documentation.
#468, #469, and #476 are parallelizable once #467 lands.

Tracking

• Land approved scheduled-benchmark contract amendments (A1–A13) #467 — F1: Land the approved contract amendments (A1–A13)
• Canonical benchmark-result/v1 schema and validator #468 — F2: Canonical result schema and validator
• Repo-owned schedule spec: manifest, thin prompt spec, labels #469 — F3: Repo-owned schedule spec (manifest, prompt spec, labels)
• Execution-side entrypoint: in-run confirmation, seal, no GitHub writes #470 — F4: Execution-side entrypoint (confirmation, seal, no GitHub writes)
• Publication CLI: sweep (validate, persist, announce, reconcile) #471 — F5: Publication CLI (sweep, validate/persist/announce/reconcile)
• Watchdog and health status for scheduled benchmark lanes #472 — F6: Watchdog and health status
• Provision benchmark-publication App, environment, rulesets, and labels #473 — F7: Provisioning specification and runbook (App, environment, rulesets)
• Provision benchmark-publication GitHub infrastructure #484 — F7: Provision and verify the benchmark-publication GitHub infrastructure (executes the Provision benchmark-publication App, environment, rulesets, and labels #473 runbook)
• Publication-only GitHub Actions workflow (schedule + workflow_dispatch) #474 — F8: Publication-only GitHub Actions workflow
• Routine provisioning, verification, and first verified benchmark runs #475 — F9: Routine provisioning, verification, and first verified runs
• Actions-side read-only validation of scheduled-benchmark contracts #476 — F10: Actions-side read-only validation
• Repository operations specification for scheduled benchmark publication #477 — F11: Repository operations specification
• GitHub Wiki page: "How the Scheduled Benchmark System Works" #478 — F12: GitHub Wiki operating-model page

Deferred conditional follow-ups (not created)

• Deferred conditional follow-up — F13: a workflow-fired Claude Routine API
  trigger to retry a missed run automatically. Condition: a maintainer
  decides the watchdog's visibility-only missed-run signal is insufficient
  and accepts adding a bearer-token environment secret for automated
  retry. Not currently true — no missed run has occurred under this design.
• Deferred conditional follow-up — F14: rotate benchmark-history to a new
  orphan branch, or move records to an external store. Condition: the
  measured packed size of benchmark-history exceeds 100 MB, or any single
  file exceeds 10 MB (the growth trigger in
  canonical-result-and-persistence.md §4). Not currently true — the
  branch does not yet exist (E1).
• Deferred conditional follow-up — F15: shard the comprehensive lane across
  more than one scheduled window. Condition: F9's (Routine provisioning, verification, and first verified benchmark runs #475) measured
  comprehensive-lane duration shows the 01:00–04:00 Israel-local completion
  window is infeasible for ~106 sequential case evaluations plus
  confirmation re-runs. Not currently true — no real run has been measured
  yet.
• Deferred conditional follow-up — F16: an event-triggered publication
  accelerator (an unprivileged push trigger on claude/benchmark-result-*
  plus a workflow_run publication job). Condition: sweep latency is shown
  to actually matter for a real operational case, and a Routine push to
  a claude/* ref has been observed to fire a GitHub Actions workflow
  (evidence E19 currently records no observed data either way). Neither
  half of this condition is currently true; the scheduled sweep remains the
  correctness path, not a Routine-triggered workflow.

Non-goals

• Re-deciding Design the production operations layer for scheduled benchmark execution #464's architecture, persistence choice, or App design.
• Any contributor/PR/merge-facing benchmark gate, Class 1 runtime, or
  GitHub Actions cron benchmark execution — permanently rejected by Design the production operations layer for scheduled benchmark execution #464
  and unchanged by this Epic.
• Creating F13–F16 ahead of their stated conditions.