You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Build the v0.1.7 core/ successor to fs-bench-pro as a fast loop for finding bugs and improving algorithms. #235 is this issue's substrate sub-issue and must land before #231 collects Init numbers. #179 is closed; its real daemon/Workspace path supplies a small diagnostic harness canary, not an Init performance result.
Migrate three prerequisite clusters before the remaining families:
#231: native init_namespace. First pass measures 100, 1,000 and 10,000 files once each. Retain the 100,000-file tier as NOT_RUN with its original cold-source and 2.7 s target unresolved. The first pass is discovery; the original four-tier completion gate remains open.
#232: the complete active SDK edit-* cluster, using the public SDK-to-live-Workspace route.
#233: tiny_file_churn and local_snapshot, including their full high-tier payload and history workloads.
Freeze and commit each family's authentic operation, case IDs, fixtures, cache stance, timer, full independent oracle, resource and numeric gates before collecting admission evidence. A selected first-pass discovery row is not an automatic PASS. Do not advance later families until all three pilots meet their final frozen gates. Keep historical receipts under their original identities.
Initial substrate contract
Start with one registered daemon-host route and the four operations list, run --case ID --out NEW, verify --run DIR, and report --run DIR. run --family init_namespace invokes the three first-pass cases serially with one build. Use one raw performance sample per case at each exact source/fixture identity; an edit creates a new diagnostic identity and fresh output. Do not add control/candidate arms, a paired scheduler, multi-sample statistics, a second Cargo workspace, or empty adapters for future routes. Add storage-direct only with a real registered C1/C2 case. #193 separately owns its matched Service direct/forward experiment; unlike routes are not subtracted into a transport number.
A real native Init must read the source directory and bytes, construct/save C1/C2 objects, and publish the C5 result inside the public Init timer. Today's InitLayerStack accepts at most 128 already-described entries with pre-saved file roots; it is not a native importer and cannot stand in for #231. Preserve the Init multi-worker exception. The M1 public workflow, M2 daemon/host delivery, M3 C1/C2 and M4 C5 report headings use actual product telemetry labels and process-local clocks; do not add overlapping spans or subtract independent clocks.
Reuse sealed fixtures and exact production binaries. One case command prepares only its selected fixture, runs one public operation, invokes a separate full verifier, parses LFT1, checks cleanup, and writes append-only evidence. Retain one exact telemetry.lft1 file per telemetry-enabled case and parsed receipt; remove only validated temporary captures owned by the run. A row with unknown source cache is discovery-only and never called cold or admission PASS. Keep every failure, ineligible result and NOT_RUN row.
Fast parallel worktrees
There is no machine-global benchmark lock and no one-active-run limit across worktrees. Each worktree owns its Cargo target, prepared fixtures, Store, scratch and output. A nonblocking worktree-local run flock prevents two runs from mutating the same worktree; builds take no benchmark lock. Use dynamic ports, unique runtime/container names and immutable executable/full image IDs. Refuse a foreign Cargo target or output path. Avoid the legacy globally locked BuildKit Cargo cache, mutable image tags and broad image pruner. Record observed CPU/disk/page-cache overlap as interference; it cannot become a clean admission result by omission.
Every invoked Cargo build, including first-use, must finish within 30 s; an overrun is BUILD_SLOW and requires a build-path fix. Every complete independent verifier child must finish within 5 s per case. Preparation, three runs, verification and cleanup should finish within 30 s per family; record and investigate a miss. Retain the established complete performance command budget of 15 s, with prospectively declared small exceptions up to 25 s. Never shorten the workload/oracle, move product work outside the timer or relax a timeout to meet these budgets.
v0.1.7 fs-bench-pro pilot 1: native init_namespace parity #231's first pass has one raw 100/1,000/10,000 observation and full separate proof per case, all nonpassing attempts retained, and the 100,000 NOT_RUN row. It does not clear the four-tier pilot gate.
Each final prerequisite pilot eventually meets its frozen complete performance, verification, cleanup and custody gate before broader migration.
The spec and benchmark AGENTS rules are committed before implementation/collection. Run only checks covering changed files; no aggregate preflight or CI claim.
Working proposal: core/docs/benchmark/fs-bench-pro/issue-231/SPEC.md; implementation handoff: core/docs/benchmark/fs-bench-pro/issue-231/HANDOFF.md; see neighboring benchmark docs. The implementation home is core/benchmark/fs-bench-pro/; do not scaffold unused paths. Source and evidence details remain governed by docs/general/benchmark_rules.md and docs/roadmap/0.1/0.1.7/measurement-isolation.md.
Goal and sequence
Build the v0.1.7
core/successor tofs-bench-proas a fast loop for finding bugs and improving algorithms. #235 is this issue's substrate sub-issue and must land before #231 collects Init numbers. #179 is closed; its real daemon/Workspace path supplies a small diagnostic harness canary, not an Init performance result.Migrate three prerequisite clusters before the remaining families:
init_namespace. First pass measures 100, 1,000 and 10,000 files once each. Retain the 100,000-file tier asNOT_RUNwith its original cold-source and 2.7 s target unresolved. The first pass is discovery; the original four-tier completion gate remains open.edit-*cluster, using the public SDK-to-live-Workspace route.tiny_file_churnandlocal_snapshot, including their full high-tier payload and history workloads.Freeze and commit each family's authentic operation, case IDs, fixtures, cache stance, timer, full independent oracle, resource and numeric gates before collecting admission evidence. A selected first-pass discovery row is not an automatic PASS. Do not advance later families until all three pilots meet their final frozen gates. Keep historical receipts under their original identities.
Initial substrate contract
Start with one registered
daemon-hostroute and the four operationslist,run --case ID --out NEW,verify --run DIR, andreport --run DIR.run --family init_namespaceinvokes the three first-pass cases serially with one build. Use one raw performance sample per case at each exact source/fixture identity; an edit creates a new diagnostic identity and fresh output. Do not add control/candidate arms, a paired scheduler, multi-sample statistics, a second Cargo workspace, or empty adapters for future routes. Addstorage-directonly with a real registered C1/C2 case. #193 separately owns its matched Service direct/forward experiment; unlike routes are not subtracted into a transport number.A real native Init must read the source directory and bytes, construct/save C1/C2 objects, and publish the C5 result inside the public Init timer. Today's
InitLayerStackaccepts at most 128 already-described entries with pre-saved file roots; it is not a native importer and cannot stand in for #231. Preserve the Init multi-worker exception. The M1 public workflow, M2 daemon/host delivery, M3 C1/C2 and M4 C5 report headings use actual product telemetry labels and process-local clocks; do not add overlapping spans or subtract independent clocks.Reuse sealed fixtures and exact production binaries. One case command prepares only its selected fixture, runs one public operation, invokes a separate full verifier, parses
LFT1, checks cleanup, and writes append-only evidence. Retain one exacttelemetry.lft1file per telemetry-enabled case and parsed receipt; remove only validated temporary captures owned by the run. A row with unknown source cache is discovery-only and never called cold or admission PASS. Keep every failure, ineligible result andNOT_RUNrow.Fast parallel worktrees
There is no machine-global benchmark lock and no one-active-run limit across worktrees. Each worktree owns its Cargo target, prepared fixtures, Store, scratch and output. A nonblocking worktree-local run
flockprevents two runs from mutating the same worktree; builds take no benchmark lock. Use dynamic ports, unique runtime/container names and immutable executable/full image IDs. Refuse a foreign Cargo target or output path. Avoid the legacy globally locked BuildKit Cargo cache, mutable image tags and broad image pruner. Record observed CPU/disk/page-cache overlap as interference; it cannot become a clean admission result by omission.Every invoked Cargo build, including first-use, must finish within 30 s; an overrun is
BUILD_SLOWand requires a build-path fix. Every complete independent verifier child must finish within 5 s per case. Preparation, three runs, verification and cleanup should finish within 30 s per family; record and investigate a miss. Retain the established complete performance command budget of 15 s, with prospectively declared small exceptions up to 25 s. Never shorten the workload/oracle, move product work outside the timer or relax a timeout to meet these budgets.Acceptance
NOT_RUNrow. It does not clear the four-tier pilot gate.Working proposal:
core/docs/benchmark/fs-bench-pro/issue-231/SPEC.md; implementation handoff:core/docs/benchmark/fs-bench-pro/issue-231/HANDOFF.md; see neighboring benchmark docs. The implementation home iscore/benchmark/fs-bench-pro/; do not scaffold unused paths. Source and evidence details remain governed bydocs/general/benchmark_rules.mdanddocs/roadmap/0.1/0.1.7/measurement-isolation.md.