Routine provisioning, verification, and first verified benchmark runs

[amirbena]

Design provenance

#464 / PR #465 — execution-publication-boundary.md §3 (scheduler adapter, provider-owned configuration, "verifying a routine exists and is running" table); decision-record.md M5; contract-reconciliation.md A11(c)/(d). F-number: F9.

Scope (exact)

Register both Claude Cloud Routines (sentinel, comprehensive) using the thin prompt spec (#469 / F3); verify each Routine's GitHub connection and schedule against the manifest; perform an observed residual-R1 check (attempt a non-claude/ push and an issue-create from a Routine smoke run, confirming no GitHub write credential beyond the seal is reachable); verify DST behavior across a transition; measure real comprehensive-lane duration (including confirmation re-runs) against the 01:00–04:00 Israel-local window and the account's daily run cap; produce the first verified sealed-and-published runs for both lanes.

As each Routine is enabled, set that lane's lanes.<lane>.expected_from in the schedule manifest (runtime_platform/benchmark/schedule-spec.md §1) to the UTC instant from which the lane is expected to run, in a manifest-only PR. It is the missed-run watchdog's (#472) reference until the lane has a published verified scheduled record; while it is null the lane is "not activated" and the watchdog never alerts for it.

Non-goals

• Any repository code change beyond the manifest-only expected_from edit (Execution-side entrypoint: in-run confirmation, seal, no GitHub writes #470, Publication CLI: sweep (validate, persist, announce, reconcile) #471, Watchdog and health status for scheduled benchmark lanes #472, Publication-only GitHub Actions workflow (schedule + workflow_dispatch) #474 already exist).
• Deciding to shard the comprehensive lane — that decision belongs to conditional follow-up F15, gated on this issue's measurement.

Dependencies

Depends on: #470, #474 (F4, F8).
Blocks: #477 (F11).
Parent: #466.

Acceptance criteria

• Both Routines exist, are enabled, and match the manifest's cadence and lane assignment.
• Residual-R1 check performed and its result (what the provider-side credential could/could not do) recorded.
• DST transition verified, or explicitly scheduled to be verified at the next transition, with the check documented.
• Comprehensive-lane duration measured and compared against the completion window; result feeds the F15 trigger decision.
• Each enabled lane's expected_from is set in the manifest (no enabled lane left null), and the health status shows no lane as not activated.
• At least one verified, published run exists per lane, visible on benchmark-history and the tracking issues.

Evidence required for completion

The merged manifest change setting expected_from for each enabled lane; links to the published records/receipts for both lanes' first runs; the residual-R1 check transcript; the measured duration figure.

Required amendment approval

None of the seven listed gates applies directly. A3 (optional scheduler-admissibility amendment) and A11 (cadence/DST wording) are Should-class and already landed via #467 (F1) if approved — this issue does not itself require new amendment approval to proceed.

Work classification

provisioning, validation

[amirbena]

Residual-R1 check: unattended Routine attempt was a NON-RESULT

Status: residual-R1 remains OUTSTANDING. The acceptance criterion "Residual-R1 check performed and its result recorded" is not met by this run.

What happened (2026-09-24):

• A temporary Cloud Routine was asked to run the R1 smoke probe unattended (credential/auth-surface inspection, a non-claude/ branch push, and a GitHub issue create).
• The unattended provider path failed closed. It classified that combination as an access probe requiring live user confirmation, and it requested that confirmation.
• No probe command was executed (steps 0–4 were all skipped).
• No GitHub write occurred: no branch was created and no issue was created.
• No bypass or rephrasing was attempted. The refusal was accepted as-is.

Interpretation:

• This is not evidence that R1 holds. A refusal to attempt the probe says nothing about whether a write credential beyond the seal is reachable.
• It is also not evidence that R1 fails.
• It only shows that unattended Routines fail closed on this probe combination.

Next step: run the R1 check in an interactive Cloud session using the same repository and GitHub connection as the production Routines, with four phases: (0) read-only baseline, (1) allowed claude/... control write, (2) non-claude/... branch probe, (3) issue-create probe. Each shared-state write gets the maintainer's explicit approval immediately before execution. If an unexpected write succeeds, it is classified as an R1 failure and the run stops before cleanup to preserve the evidence. The transcript will be recorded here.

No changes to #475's acceptance criteria.

[amirbena]

Residual R1 state: unobservable through the supported provider execution path

Residual-R1 is neither PASS nor FAIL. #475's R1 acceptance criterion remains OPEN.

Two attempts were made to run the R1 smoke probe through the supported Cloud Routine execution path (2026-09-24). Both failed closed:

1. Unattended Routine run. The Routine classified the combination of credential/auth-surface inspection, a non-claude/ branch push and a GitHub issue create as an access probe requiring live user confirmation. It executed none of the probe steps.
2. Follow-up interactive authorization attempt. The maintainer sent live confirmation in the resulting session. The Routine refused to treat it as valid authorization, because the session originated as an unattended scheduled task.

Facts for the record:

• Neither attempt is a PASS or a FAIL.
• No prohibited write was attempted. No non-claude/ push was tried, no issue was created, and no branch was created.
• No credential exploration occurred, and no bypass or rephrasing was attempted.
• The underlying provider credential capability therefore remains unobserved.
• The supported Routine execution path prevents reaching the credential boundary this check is meant to test.

Governing text is unchanged. runtime-execution-contract.md still requires the observed scope of the provider-native identity (R1) to be verified before relying on unattended operation, and no amendment has been made.

Next: the maintainer will collect read-only configuration evidence (GitHub App installation, repository access, granted permissions, Routine connection details). That is mitigation / capability-ceiling evidence only. It does not satisfy R1.

[amirbena]

Residual R1: capability-ceiling evidence (read-only configuration audit)

This is mitigation / capability-ceiling evidence only. It does not satisfy behavioral R1. R1 remains "unobservable through the supported provider execution path". Neither earlier refusal is a PASS or a FAIL. #475's R1 acceptance criterion remains open.

The audit was read-only. No GitHub or Claude settings were changed, and no probe was run.

GitHub

• Claude GitHub App (Anthropic) is installed on the maintainer's personal account.
  • Repository access: All repositories.
  • Permissions shown by GitHub:
    • read: commit statuses, metadata;
    • read/write: actions, checks, code, discussions, issues, pull requests, repository hooks, workflows.
• Consequence: at the App-installation level, code writes and issue writes are permitted. If the Routine's Git credential derives from this App, the GitHub-side installation permissions alone do not enforce R1. A narrower provider/session proxy may still do so, and that is unverified.
• Claude for GitHub is a separate installation. It has read-only access to administration, code and metadata. It cannot by itself explain the benchmark seal write path.

Claude Routines UI

• Sentinel Routine: Remote, repository amirbena/code-review-skill, no connectors.
• Temporary R1 Probe Routine: Remote, same repository, no connectors.
• The UI does not expose which GitHub App or identity backs the Routine's Git credential.
• Therefore this audit does not establish that the production Routines use the Claude GitHub App, and no such claim is made.

Conclusion

• The exact Routine credential identity and its effective provider-side restrictions remain unverified.
• If the credential derives from the Claude App, the GitHub-side ceiling is broad (Contents and Issues read/write on all repositories), and any restriction to claude/ refs and no issue writes would rest entirely on the provider proxy.
• The governing contract is unchanged.