Skip to content

v0.1.7: migrate fs-bench-pro with daemon-host and storage-direct modes #230

Description

@yifanxuaaa

Goal and sequence

Build the v0.1.7 core/ successor to fs-bench-pro as a fast loop for finding bugs and improving algorithms. #235 is this issue's substrate sub-issue and must land before #231 collects Init numbers. #179 is closed; its real daemon/Workspace path supplies a small diagnostic harness canary, not an Init performance result.

Migrate three prerequisite clusters before the remaining families:

  1. #231: native init_namespace. First pass measures 100, 1,000 and 10,000 files once each. Retain the 100,000-file tier as NOT_RUN with its original cold-source and 2.7 s target unresolved. The first pass is discovery; the original four-tier completion gate remains open.
  2. #232: the complete active SDK edit-* cluster, using the public SDK-to-live-Workspace route.
  3. #233: tiny_file_churn and local_snapshot, including their full high-tier payload and history workloads.

Freeze and commit each family's authentic operation, case IDs, fixtures, cache stance, timer, full independent oracle, resource and numeric gates before collecting admission evidence. A selected first-pass discovery row is not an automatic PASS. Do not advance later families until all three pilots meet their final frozen gates. Keep historical receipts under their original identities.

Initial substrate contract

Start with one registered daemon-host route and the four operations list, run --case ID --out NEW, verify --run DIR, and report --run DIR. run --family init_namespace invokes the three first-pass cases serially with one build. Use one raw performance sample per case at each exact source/fixture identity; an edit creates a new diagnostic identity and fresh output. Do not add control/candidate arms, a paired scheduler, multi-sample statistics, a second Cargo workspace, or empty adapters for future routes. Add storage-direct only with a real registered C1/C2 case. #193 separately owns its matched Service direct/forward experiment; unlike routes are not subtracted into a transport number.

A real native Init must read the source directory and bytes, construct/save C1/C2 objects, and publish the C5 result inside the public Init timer. Today's InitLayerStack accepts at most 128 already-described entries with pre-saved file roots; it is not a native importer and cannot stand in for #231. Preserve the Init multi-worker exception. The M1 public workflow, M2 daemon/host delivery, M3 C1/C2 and M4 C5 report headings use actual product telemetry labels and process-local clocks; do not add overlapping spans or subtract independent clocks.

Reuse sealed fixtures and exact production binaries. One case command prepares only its selected fixture, runs one public operation, invokes a separate full verifier, parses LFT1, checks cleanup, and writes append-only evidence. Retain one exact telemetry.lft1 file per telemetry-enabled case and parsed receipt; remove only validated temporary captures owned by the run. A row with unknown source cache is discovery-only and never called cold or admission PASS. Keep every failure, ineligible result and NOT_RUN row.

Fast parallel worktrees

There is no machine-global benchmark lock and no one-active-run limit across worktrees. Each worktree owns its Cargo target, prepared fixtures, Store, scratch and output. A nonblocking worktree-local run flock prevents two runs from mutating the same worktree; builds take no benchmark lock. Use dynamic ports, unique runtime/container names and immutable executable/full image IDs. Refuse a foreign Cargo target or output path. Avoid the legacy globally locked BuildKit Cargo cache, mutable image tags and broad image pruner. Record observed CPU/disk/page-cache overlap as interference; it cannot become a clean admission result by omission.

Every invoked Cargo build, including first-use, must finish within 30 s; an overrun is BUILD_SLOW and requires a build-path fix. Every complete independent verifier child must finish within 5 s per case. Preparation, three runs, verification and cleanup should finish within 30 s per family; record and investigate a miss. Retain the established complete performance command budget of 15 s, with prospectively declared small exceptions up to 25 s. Never shorten the workload/oracle, move product work outside the timer or relax a timeout to meet these budgets.

Acceptance

Working proposal: core/docs/benchmark/fs-bench-pro/issue-231/SPEC.md; implementation handoff: core/docs/benchmark/fs-bench-pro/issue-231/HANDOFF.md; see neighboring benchmark docs. The implementation home is core/benchmark/fs-bench-pro/; do not scaffold unused paths. Source and evidence details remain governed by docs/general/benchmark_rules.md and docs/roadmap/0.1/0.1.7/measurement-isolation.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions