Skip to content

feat(buck2): add standalone root and aggregate gates - #1301

Merged
schickling merged 24 commits into
mainfrom
schickling-assistant/2026-09-15-quick-aggregate
Sep 23, 2026
Merged

schickling merged 24 commits into
mainfrom
schickling-assistant/2026-09-15-quick-aggregate

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Problem

Effect-utils had a tracked standalone Buck root, but ordinary CI still synthesized a self-composition and several aggregate gates retained composition-era prerequisites. A fresh standalone checkout also attempted Genie freshness before publishing the committed-graph dependency views its generator sources require.

Goal

Make the tracked checkout the single normative root for ordinary development and single-repository CI, remove effect-utils CI self-composition, and keep the transition contract explicit in the Buck VRS.

Decisions

  • COMP-T01: external consumers and single-repository CI build public members from tracked standalone roots; trust tiers may use separate cache namespaces.
  • COMP-R01: the tracked checkout is the normative Buck project root; only explicitly requested cross-repository work may synthesize the paused composed shape.
  • COMP-R02: canonical repos/<name> mounts apply only inside that explicit composed shape; the standalone repository maps its canonical cell to ..
  • COMP-R06: standalone repositories carry .buckroot; nested member markers do not override an explicitly composed outer root.
  • COMP-R07: every supported root shape has a fixed isolation directory; cross-shape action-key parity is not promised and per-invocation isolation is forbidden.
  • BUCK-R06: zero re-execution is required between identical same-platform standalone checkouts; the interpretation now cites the second-context key-stability experiment, while standalone-versus-composed comparison remains migration evidence rather than a permanent portability contract.
  • Decision 0020 Amendment 4: the one-writable-mount workspace is limited to explicitly requested cross-repository compositions during retirement; effect-utils CI runs from the standalone checkout.
  • Decision 0027 Amendment 1: standalone worktrees are again the default for ordinary development, agents, and single-repository CI; composed workspaces are an explicit temporary exception.
  • CI no longer runs mr:apply, mr:setup, or composition cleanup. Its checkout is the source root for every lane.
  • genie:check runs committed-graph editor bootstrap followed by an editor-generator closure precheck, preserving mutation-free freshness while proving every first-party runtime import is covered.
  • TypeScript declaration publication accepts the tracked standalone root only when .buckroot is present and Git reports that same directory as the repository top level. The paused composed compatibility path still requires reciprocal worktree identity; lookalike repos/effect-utils directories fail closed.
  • Megarepo command preflight classifies a Git-top-level checkout as standalone only when tracked .buckroot and .buckconfig markers are present and no repos/ mount exists; untracked marker lookalikes and mounted compositions retain the fail-closed legacy checks.
  • Editor bootstrap materializes the declared Genie generator import closure (the repository root and OpenTelemetry contract views), emits phase timings, preserves whole-workspace authority checks, and overlaps warm integrity traversals instead of serializing duplicate snapshots.
  • CI publishers are scoped to the views each lane consumes, while coalesced Restate, OpenTelemetry-contract, and Playwright publishers retain declared dependency authority.
  • CI task traces and cache-reuse diagnostics survive the standalone-root cutover.
  • DELTA-002 records the q66-accepted warm-integrity budget: a 62.6-second fresh two-view publication followed by 8.61, 7.18, and 7.76-second warm samples.
  • Every standalone Buck entry reconciles the exact BUCK2_NO_REMOTE_CACHE=1 trust posture into a gitignored .buckconfig.local before Buck starts, disabling remote reads and uploads without replacing trusted-context coordinates.
  • The second-context experiment resolves and removes DELTA-001: after warming the identical revision, the sandbox reused 1,192 of 1,195 queried actions; the sole local action was the same failing typecheck with an identical digest, proving the earlier 633-local sample was test ordering rather than key instability.
  • docs(buck2): resolve second-context reuse delta #1322 is superseded by this PR; its final key-stability record and delta removal are folded into the VRS commit here.
  • The paused composed shape and its guards remain only for consumers that still explicitly request cross-repository composition; deletion belongs to L3 cut 2.

Verification

  • Pinned Genie --check: all 155 generated files unchanged after folding the bootstrap fix.
  • Focused Bun suites for Buck2 projection and GitHub workflow generation: 188 passed, 6 skipped, 0 failed.
  • Strict Buck VRS validation reaches only the eight pre-existing metadata errors in decisions 0035/0036.
  • Standalone task and materializer guards pass, including tracked-root acceptance, reciprocal composed-root acceptance, lookalike rejection, and a graph with no composition prerequisite on ordinary Buck tasks.
  • A pinned Buck run from a detached local-disk checkout passed effect_utils//packages/@overeng/utils:test: 25 files, 233 tests, and 528 local commands with no remote execution or cache use. The daemon connected in about 0.86 seconds and the Buck work completed in about 19 seconds.
  • The shallow stage-0 source-input proof passes with the explicit shallow flake reference.
  • Standalone cache-posture integration proof: 3 passed, 0 failed; exact 1 disables reads/uploads, non-exact values preserve trusted defaults, and existing local overrides survive managed-block removal.
  • The folded second-context record preserves VRS shape and documents identical action digests across normal and sandboxed contexts.
  • The folded bootstrap branch passed its clean local-checkout proof before integration; its six commits remain reviewable below the VRS commit.
  • q66 closure proof: direct Weaver load passed; the checker accepted all 158 closures and the root-only negative named otel-contract; focused units passed 41 tests/212 expectations; editor integration passed 20 tests/102 expectations.
  • The standalone Megarepo classifier passed its local-disk integration suite: 1 file and 7 tests, covering untracked-marker rejection, tracked standalone acceptance, and repos/-mount rejection.
  • Pinned Buck passed effect_utils//packages/@overeng/megarepo:test from the local-disk checkout: 34 files, 625 tests, and 612 local commands; a real mr --cwd . status --output json invocation also exits 0 at the standalone root.
  • The unbounded OTEL-contract lane passes after excluding the standalone root's buck-out and target trees from the repository seam scan.
  • The task-graph checker passes both metadata modes: summarized tasks list --json proves each scoped/full publisher retains executable authority, while the realized registry proves trace identity, canonical package scope, and the whole-workspace unscoped fallback.
  • The untouched Restate timing lane passed its one local-disk rerun after the Linux CI flake.
  • The nested real-otelite capture passes while a simulated outer CI spool is present: it explicitly isolates nested delivery, then observes the task, compiler-command, and TypeScript phase spans in one trace with child exit 0.
  • The downstream bounded aggregate passes from the local-disk checkout: buck2:quick includes the new Weaver targets and Megarepo typecheck, completing 1,212 commands with no remote execution.
  • Required GitHub checks completed successfully in run 35798036377.

Complexity

  • Ledger: 1,120 lines added, 321 removed, net +799, measured by buck2-ledger.ts at b492ec4b66152fa3193196152b78b48baa33121c.
  • Twenty-four commits keep the standalone graph, CI cutover, bootstrap/visibility changes, q66 closure fix and DELTA-002, Megarepo standalone classification and typed probe boundary, repository-scan isolation, cross-mode task metadata checks, nested OTEL capture isolation, and final VRS amendments reviewable in order.

Concerns

  • Public pull-request lanes are intentionally cache-cold until dotfiles#2986 deploys the public read-only tier; that public address will replace dev3 for public lanes. The trusted remote-cache proof remains main-only and must not be weakened on this PR.
  • Cross-shape action-key parity is deliberately not a long-term promise.

Friction & bottlenecks

  • Measurements from /srv/bulk are invalid for Buck budget decisions because dotfiles#2955 makes filesystem walks pathological (roughly 60 seconds for the pruned tree versus 0.17 seconds on local disk). The local-disk control run above proves daemon startup and the representative test target without that host artifact.
  • Run 35729756932 exposed four independent clean-CI defects: missing exported-symbol JSDoc, an over-broad standalone-root lookalike, a shallow-flake revCount read, and a Storybook fixture that assumed node on PATH. All four are fixed at the replacement head.
  • q66 folds the functional generator-closure fix and public-safe DELTA-002 record into this PR. Run 35798036377 exercises the replacement head b492ec4b66152fa3193196152b78b48baa33121c.

Follow-ups

  • Delete the remaining explicitly composed effect-utils shape when downstream edges flip at L3 cut 2.
  • Complete the separate public cache tier before expecting the trusted main-only proof lane to pass.

References

  • context/buck2/05-composition/requirements.md
  • context/buck2/.decisions/0020-one-writable-mount-workspaces.md
  • context/buck2/.decisions/0027-composed-default-worktrees.md
  • context/buck2/requirements.md
  • context/buck2/.experiments/2026-09-19-second-context-key-stability.md
Posted on behalf of @schickling
field value
agent_identity dev3.direct.omp.t3mkm4sd
session unknown
agent_persona generalist
agent_supervisor unavailable
agent_tool OMP
agent_tool_version 18.1.14
agent_runtime OMP 18.1.14
tooling_profile dotfiles@5d1e05f

@schickling-assistant
schickling-assistant added this pull request to stack #1270 September 18, 2026 07:10
@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-09-15-quick-aggregate branch from 3e3890a to 960ff69 Compare September 18, 2026 08:34
@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-09-15-quick-aggregate branch 2 times, most recently from 948d397 to 348c000 Compare September 19, 2026 19:13
Base automatically changed from schickling/2026-09-12-buck2-js-ts-closure to main September 20, 2026 06:08
@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-09-15-quick-aggregate branch 2 times, most recently from 3b50809 to 1aaceac Compare September 21, 2026 22:05
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
Keep the bootstrap package set explicit because the TypeScript import walker itself is unavailable until the root editor view exists. Reuse that walker immediately after publication as a genie:check prerequisite, reporting every first-party runtime boundary and naming any workspace view omitted by the declaration. This preserves the two-view R07 hot path while making a future generator import fail during authoring instead of only in a clean CI checkout.

agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
Attribute a runtime boundary to the workspace package that imports it, with the repository root view explicitly backed by Genie's package tree. This reports the missing otel-contract view for the Weaver content-address edge without incorrectly requiring a separate content-address publication.

agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
Treat bare packages already supplied by Genie's root package tree as covered before requiring an importer-specific editor view. The guard now reports otel-contract for the content-address edge while accepting megarepo and ci-tools imports whose boundaries are already available from the root view.

agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
Fingerprint the immutable backing and node_modules payload roots concurrently while preserving their deterministic framing order. Both complete integrity checks still run; only their I/O overlap changes.

agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
Publish bootstrap views that use different editor-root locks concurrently while preserving deterministic serial order inside each shared lock. Root and workspace-package closure validation now overlap without weakening either snapshot proof.

agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
Record q66's accepted 7.2–8.6 second two-view warm residual against BUCK-R07's five-second budget. Resolution requires a decision on a persisted recursive metadata integrity proof before changing the editor-view record contract.

agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-09-15-quick-aggregate branch from 1131411 to 4ab6319 Compare September 22, 2026 21:03
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
agent-identity: dev3.direct.omp.t3mkm4sd
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.14
agent-runtime: OMP 18.1.14
tooling-profile: dotfiles@5d1e05f
@schickling-assistant
schickling-assistant marked this pull request as ready for review September 23, 2026 00:33
@github-actions
github-actions Bot requested a review from schickling September 23, 2026 00:33
@schickling-assistant

Copy link
Copy Markdown
Collaborator Author

@codex Please review this pull request for merge readiness, with particular attention to the standalone-root contract, CI cutover, and the documented VRS amendments.

Posted on behalf of @schickling
field value
agent_identity dev3.direct.omp.fkazdy6v
session unknown
agent_persona generalist
agent_supervisor unavailable
agent_tool OMP
agent_tool_version 18.2.6
agent_runtime OMP 18.2.6
agent_model openai-codex/gpt-5.6-sol
worktree 2026-09-22-fix-checks/2026-09-22-fix-checks
tooling_profile dotfiles@965ef7e-dirty

@schickling
schickling merged commit c25fb32 into main Sep 23, 2026
28 checks passed
@schickling
schickling deleted the schickling-assistant/2026-09-15-quick-aggregate branch September 23, 2026 00:37

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b492ec4b66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread devenv.nix
Comment on lines +1368 to +1371
exec "$BUCK2_BIN" audit providers \
--target-platforms //buck2/platforms:host_platform \
//buck2/toolchains:cross_cell_provider_identity \
//buck2/toolchains:cross_cell_product_identity

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Route the typecheck job through the aggregate

In the CI typecheck context (.github/workflows/ci.yml.genie.ts:412) and the documented standalone command, buck2:check is invoked directly, but this new implementation only audits two providers. The admitted TypeScript targets moved to buck2:quick, whose dependency points from buck2:quick to buck2:check, so invoking buck2:check never reaches them; a PR containing TypeScript errors can therefore receive a green required “Type check (Buck)” result. Keep the TypeScript build behind this command or update every direct caller to use the aggregate.

AGENTS.md reference: AGENTS.md:L7-L10

Useful? React with 👍 / 👎.

Comment on lines +94 to +98
This experiment resolves DELTA-001 as a test-ordering artifact: S8's sandbox
run was the first build at its rebased revision; at one revision with the
normal context warmed first, the sandbox reused 1,192 of 1,195 queried actions,
and the remaining action failed identically in both contexts. DELTA-001 is
removed. BUCK-R06 and REUSE-R02 remain unchanged.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not close the reuse delta with a red replay

This evidence does not satisfy the cited BUCK-R06/REUSE-R02 acceptance condition: both //:quick builds exited nonzero, and the second context still executed one action locally, whereas the VRS requires zero locally executed actions for an unchanged warmed target (context/buck2/04-reuse/spec.md:46). Equal digests diagnose the original cache misses, but a failing action cannot populate the successful cache entry needed to test the zero-reexecution contract, so removing DELTA-001 here leaves the requirement without its claimed proof; retain the delta until the same experiment completes green with zero local actions.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-23T00:41:08.310975Z b492ec4 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants