[Feature] Publish a portable reviewed source baseline for the existing ProgramBench harness

[rajesh-ms]

Outcome

Dependent benchmark fixes target a maintainer-reviewable immutable source commit containing the existing R4 harness, remote adapter, unlimited-budget support and sanitized synthetic fixtures, rather than code available only on one machine.

Observed behavior

Original base is b2523964e7576cafc00e84a51e1044f55826dea7. R4 used an uncommitted 618-file local snapshot, archive SHA-256 6ce23a6e8eb433a4965001a566c4198f7b40140d691003d1e44707a833dacb1b. Capability/ProgramBench helpers, Python tests and registrations are absent from that base; tracked core/web files also contain local remote-adapter/unlimited-budget changes. The later completion observer is an output-only helper outside the source archive.

Expected behavior

Curate the required existing changes into an accessible portable baseline. Preserve the measured archive and old receipts unchanged. Local files/archive hashes must not be presented as public Git commits.

Reproduction and evidence

Check out the original base and compare availability against the sanitized source-hook manifest. Full findings: #296 (comment); overview: #296. Historical local test results are context, not validation of the eventual published commit.

Acceptance criteria

• Inventory and review the exact local change set, source hashes, runtime/licensing prerequisites and portable configuration; provide the reviewed source branch delivered by the source-intake issue and the exact resulting commit SHA recorded in source-intake acceptance evidence accessible to downstream workers.
• Exclude personal paths, secrets/provider configuration, candidates, held-out bodies and raw traces. Publish sanitized metadata and synthetic fixtures; do not upload the raw archive blindly.
• Port required helpers/tests and the completion observer to repository-owned paths. The proposed tools/capability_completion_observer.py must not depend on a historical PID/private output file.
• Preserve Unlimited -1, zero provider-free checkpoint and null/default finite semantics where applicable, plus cost/loop/tenant/Corp/requester authority and cancellation safety.
• Run exact contributor gates and targeted harness/remote-adapter/unlimited-budget tests on the final commit; distinguish reused historical runtime evidence from new checks.
• An independent reviewer confirms the available immutable baseline and source mapping; local archive presence alone does not close this issue.

Dependencies

Blocked by #298.

Aligned with #164, #53 and #166. Preserve their distinct scope and existing recovery/authority contracts. Review PR #294 before touching verifier behavior.
This serialization prevents the source intake from overwriting the preceding core handoff/deadline fixes. The source publication remains a prerequisite for benchmark-specific work.

Governed delivery contract

• Source repository: All-The-Vibes/ecorp.
• Source base: original commit above plus the preserved local snapshot; curator selects the current main/authorized parent and records the resulting exact commit.
• Write scope: only the named capability/ProgramBench helpers and corresponding tests; remote adapter crates/crony-runner/src/adapter/codex_remote.rs and required adapter wiring; token-budget domain/protocol/runner/server/store/CLI/web changes; migration 0042_unlimited_token_budgets.sql and manifest; portable synthetic capability fixtures; associated architecture/unlimited/ProgramBench docs. Curator must turn this inventory into an exact allowlist before dispatch. Exclude output/**, private configuration, candidate worktrees and held-out data.
• Provider/strategy and budgets: curate explicit native harness/model/strategy and unlimited tokens (-1), using the qualified native runtime, 10,000,000 microusd ($10) per implementation mission, subject to existing requester/Corp limits, no inherited 20-minute benchmark cutoff; one initial task attempt, scoped work and native cost/loop/stop controls; report insufficient authority rather than extend it; coding execution requires native Factory eligibility, a durable claim and the reviewed source package.
• Allowed tools: source intake/portability edits and declared checks in an isolated worktree; no unrelated uncommitted work or R4 modifications.
• Persisted verifier: final-checkout contributor gates, targeted adapter/budget/observer tests and required owned full-stack evidence; independent source review.
• Publication: reviewed source PR or explicitly linked coherent intake slices, ending in one immutable downstream baseline; no merge, auto-merge or deployment authority.

Harness-native-first and boundaries

Retain thin adapters around the pinned native Codex and ProgramBench APIs; inspect supported session, permission, artifact and verification capabilities before changing mechanisms. S packages/reconciles the existing baseline; the six functional improvements belong in their own issues. Missing prerequisites or excessive scope remain explicit blockers rather than an excuse to sweep in the whole workspace.

Validation

Run node tools/check_migrations.mjs, cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, pnpm build:web, pnpm lint:web, plus targeted capability/packaging/remote-adapter/unlimited-budget checks. Historical 177 capability, 21 packaging and nine journal tests are not final-commit claims. Record exact commands, counts, source/environment hashes, failures and hosted-CI status.

Findings attachment

The full sanitized investigation, controls, quantitative findings, source-provenance hashes and TF-01–16 acceptance matrix are attached inline in #296. This issue includes its relevant observations above. No private local path is required to read the findings.

[rajesh-ms]

Attached source-availability manifest for #299. These are hashes and availability records for the inspected local source; they do not assert that unpublished files exist in the public base. Core fixes #297/#298 can use existing main. The later benchmark fixes require the reviewed source baseline described here.

Sanitized source-hook manifest (19 entries)

{
  "schema_version": 1,
  "source_repository": "All-The-Vibes/ecorp",
  "original_base_commit": "b2523964e7576cafc00e84a51e1044f55826dea7",
  "r4_local_uncommitted_source_archive_sha256": "6ce23a6e8eb433a4965001a566c4198f7b40140d691003d1e44707a833dacb1b",
  "availability_note": "Path existence in a local checkout or source archive does not assert publication on GitHub. Core-only FX01/02 target base-native paths; S must publish portable source and fixtures for FX03/04/05/06. The output-only completion observer needs a new portable repository location.",
  "hooks": [
    {
      "path": "crates/crony-server/src/planning.rs",
      "present_in_original_base": true,
      "local_status": "locally modified tracked file",
      "current_local_sha256": "21bee152866b198dbe1cfdc7a24ceaeed6cba204dcfc61810e908d38490dc57f",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "21bee152866b198dbe1cfdc7a24ceaeed6cba204dcfc61810e908d38490dc57f"
    },
    {
      "path": "crates/crony-server/src/main.rs",
      "present_in_original_base": true,
      "local_status": "locally modified tracked file",
      "current_local_sha256": "3a1b9228ba768505603547af1144ed85ffa676b35942f89ff850d4e8de40272a",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "3a1b9228ba768505603547af1144ed85ffa676b35942f89ff850d4e8de40272a"
    },
    {
      "path": "crates/crony-server/src/dependency_source.rs",
      "present_in_original_base": true,
      "local_status": "unchanged tracked file",
      "current_local_sha256": "e0bf11114a2eafaaac93929729ff0411743e75095cd720c96c3487e070796400",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "e0bf11114a2eafaaac93929729ff0411743e75095cd720c96c3487e070796400"
    },
    {
      "path": "crates/crony-domain/src/lib.rs",
      "present_in_original_base": true,
      "local_status": "locally modified tracked file",
      "current_local_sha256": "5c37d1e824a5a55b7114b481a01f1a4ca4c6f2b0fdb47a1fec48b33bb59147a4",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "5c37d1e824a5a55b7114b481a01f1a4ca4c6f2b0fdb47a1fec48b33bb59147a4"
    },
    {
      "path": "crates/crony-store/src/lib.rs",
      "present_in_original_base": true,
      "local_status": "locally modified tracked file",
      "current_local_sha256": "e268947104235ed640198f93baac32536472ea779516f3852865c7e6019b93bf",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "e268947104235ed640198f93baac32536472ea779516f3852865c7e6019b93bf"
    },
    {
      "path": "crates/crony-runner/src/main.rs",
      "present_in_original_base": true,
      "local_status": "locally modified tracked file",
      "current_local_sha256": "543bcfebd7bccc66fcff9820a3e9619b960420e151dd886141e72ce89a915745",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "543bcfebd7bccc66fcff9820a3e9619b960420e151dd886141e72ce89a915745"
    },
    {
      "path": "crates/crony-runner/src/verifier.rs",
      "present_in_original_base": true,
      "local_status": "unchanged tracked file",
      "current_local_sha256": "943975435330880d6e10d7607255230e6e91b497de73cca9235a4ee2ea7c66a4",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "943975435330880d6e10d7607255230e6e91b497de73cca9235a4ee2ea7c66a4"
    },
    {
      "path": "crates/crony-runner/src/adapter/codex.rs",
      "present_in_original_base": true,
      "local_status": "locally modified tracked file",
      "current_local_sha256": "79d86f92242e90427a8214b4a8532849443ce97640b5dcd4194d2eb471f4b5ea",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "79d86f92242e90427a8214b4a8532849443ce97640b5dcd4194d2eb471f4b5ea"
    },
    {
      "path": "crates/crony-runner/src/adapter/codex_remote.rs",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "dccd536a9e0964bbb46cef823b236abd091543e10c214071097a316bd896aa4b",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "dccd536a9e0964bbb46cef823b236abd091543e10c214071097a316bd896aa4b"
    },
    {
      "path": "tools/capability_campaign.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "acf20cc45386a5a25b4a24f72f3d04a17496cda334d3cc86558e6b10881cc218",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "acf20cc45386a5a25b4a24f72f3d04a17496cda334d3cc86558e6b10881cc218"
    },
    {
      "path": "tools/capability_ecorp.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "b73600796d06759422127569204a412b334079f3d9310fb509fb4af1c14e2bda",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "b73600796d06759422127569204a412b334079f3d9310fb509fb4af1c14e2bda"
    },
    {
      "path": "tools/capability_programbench.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "d4acd9d74d2a1db8e27846b39d832ed149bd74adf49e0ca4dba09cd6ac9219dc",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "d4acd9d74d2a1db8e27846b39d832ed149bd74adf49e0ca4dba09cd6ac9219dc"
    },
    {
      "path": "tools/capability_native.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "7220835afe00832b6d02a5dbee7a81edeb88b5df938b0c40a921540616ed139c",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "7220835afe00832b6d02a5dbee7a81edeb88b5df938b0c40a921540616ed139c"
    },
    {
      "path": "tools/capability_benchmark_report.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "88cfeca553964b4b30058b20c2cc6dbbb9001fdd041db1f6bd4c2ce95cc387e2",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "88cfeca553964b4b30058b20c2cc6dbbb9001fdd041db1f6bd4c2ce95cc387e2"
    },
    {
      "path": "tools/capability_eval_db.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "a1f791c47b4a31c9ba3966256d39e5693278338fa2760b7688ac9608ca7f525a",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "a1f791c47b4a31c9ba3966256d39e5693278338fa2760b7688ac9608ca7f525a"
    },
    {
      "path": "tools/capability_eval_dashboard.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "bbb6d48a0024555dff32cd3e97416dc5f7942539c550ace2d29678fba532f6bb",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "bbb6d48a0024555dff32cd3e97416dc5f7942539c550ace2d29678fba532f6bb"
    },
    {
      "path": "tools/programbench_report.py",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "589016893bf968c501368b11fa7bd23c779ac0b3cf3b787a73926ecbaa39cce3",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "589016893bf968c501368b11fa7bd23c779ac0b3cf3b787a73926ecbaa39cce3"
    },
    {
      "path": "tests/capability/record.schema.json",
      "present_in_original_base": false,
      "local_status": "local untracked addition",
      "current_local_sha256": "e2dada3aa52f584ca23301800e0029179362c2ff74713911203ab151a37e3bd6",
      "in_r4_source_snapshot": true,
      "r4_source_snapshot_sha256": "e2dada3aa52f584ca23301800e0029179362c2ff74713911203ab151a37e3bd6"
    },
    {
      "path": "output/capability-benchmark-20260915-r4/completion_observer.py",
      "present_in_original_base": false,
      "local_status": "local output helper",
      "current_local_sha256": "cd029974f6b5b29986ce726f5e310da284ce3800d337bd16497d04bd1251d5ef",
      "in_r4_source_snapshot": false,
      "r4_source_snapshot_sha256": null
    }
  ]
}