ci(test): a file-level slice builds its dependency closure in a passthrough-free run, so the sharded leg stops rebuilding it by claude[bot] · Pull Request #16868 · objectstack-ai/objectstack

claude · 2026-09-08T12:28:33Z

Root cause, measured — the rebuild is real, and one half of it is removable

The card's lead was right and this PR removes the removable half of it.

turbo folds a run-level passthrough into the hash of every task in the run, not
only the task that receives it. Run this shard's tests gives a file-level slice its own
invocation precisely so that -- "--shard=k/n" reaches only that package's vitest — and
that passthrough then re-hashed the slice's entire ^build closure.

Measured on turbo 2.10.10, --filter=@objectstack/cli, turbo run test … --dry=json
(60 tasks: 59 build + 1 test):

plans compared	identical hashes	changed
plain vs plain — control	60	0
plain vs `-- --shard=1/2`	0	60
`--shard=1/2` vs `--shard=2/2`	0	60

So the sliced leg could hit neither the main-seeded Turbo cache restored earlier in the
job nor the builds the shared leg had run seconds before, in the same job, on the same
disk. Shards 5/6 and 6/6 — the only two that carry a slice — built that closure twice per
job, unconditionally.

Live corroboration, Test Core (5/6) of run 34193080219 (a packages/spec PR, so the
affected set reaches @objectstack/cli): the slice leg reported
Cached: 2 cached, 58 total and Time: 9m18.941s, while @objectstack/cli:test itself
measured Duration 187.49s. ⇒ roughly six of those nine minutes were the duplicate
rebuild, not tests. #16395's cancelled job died mid-stream through @objectstack/cli's suite — that is this leg.

The change

Two edits to .github/workflows/ci.yml, both inside the test job:

A new step, Build the sliced package's dependency closure, builds the slice's
closure in a run that carries no passthrough — turbo run build --filter=$PKG.
Measured: 59 build tasks, all 59 hash-identical to the ones in the passthrough-free
test plan (0 differing, 0 extra, 0 missing against the test's own ^build closure), so
they replay from the restored cache / the shared leg instead of re-executing.
--filter=...^$PKG was measured too and schedules 12 packages the closure does not need.
The slice's test leg gains --only, so the poisoned hashes reach exactly one task.
Verified: the --only plan is 1 -> @objectstack/cli#test.

It is its own step rather than a second guarded run inside the existing one because a
guarded site is the triple (file, job, step): measure-stall-guard-headroom refuses to
report a verdict when two guarded runs share one, and refusing is right — the two would be
judged against the worst reading of their union. That gate went red on the one-step shape
and green on this one; it named the fix and this is the fix it named.

A shard with no slice is behaviourally unchanged — the new step runs zero iterations
and the test step's shared leg is byte-identical.

How I convinced myself this is correct beyond "the gates passed"

CI configuration is not executed by the gates, so the evidence is separate:

The workflow parses and both step scripts are valid shell. The test job's YAML is
parsed, the two run: bodies extracted, and each passes bash -n. Step order is
re-read from the parse: the attestation pair is still the job's last two steps (CI 聚合门禁把合并队列重建的 aggregate result: abandoned 判成红 —— 在队 PR 零测试失败被踢出(ci.yml 两处白名单缺 abandoned) #6082).

An offline behavioural harness drives the extracted step bodies with a stub
run-with-stall-guard.mjs that records argv, over four shard shapes:

shape	observed
shard with a slice	build (no passthrough) → shared leg → `--only` sharded leg; `test-core.log` carries exactly the 2 test legs, not the build log
shard without a slice	shared leg only — no build run, argv unchanged
empty shard file	both steps short-circuit, exit 0
slice build fails	build step exits with the command's real status (17); the test step never runs ⇒ no attestation ⇒ fail-closed

The replay this PR depends on was executed, not assumed. Back-to-back invocations of
the exact new command: first 5 cached, 57 total / Time: 3m45.918s; second
57 cached, 57 total / Time: 153ms >>> FULL TURBO. A passthrough-free build run in a
second turbo invocation replays completely — which is exactly what the poisoned leg
could not do.
The control fires. The hash instrument was run twice with identical arguments and
returned 60/60 identical hashes before it was used to claim 0/60.

Re-taken timing on current `main` (the card's numbers are from 2026-09-06)

timeout-minutes for this job is 45, not 30 — #16173 raised it temporarily, with an
explicit revert condition tied to its own shard rebalance. This PR ⛔ does not touch that
line: raising the ceiling is #16395's fallback remedy, which is not being taken, and
lowering it is #16173's declared condition, not this PR's.

Test Core (5/6) on three packages/spec-touching PRs, current main:

PR	run	`Test Core (5/6)`	Build Core
#16846	`34215028203`	success, 14.8 min	5.3 min
#16823	`34205918018`	success, 22.3 min	5.9 min
#16800	`34193080219`	success, 22.6 min	5.8 min

⇒ the acute 30-minute blockage the card describes is gone; the defect it was a symptom of
was not, and is what this PR removes.

Acceptance notes

A3 checked and it changes the reading: shard 5/6 does not carry
@objectstack/spec — spec is on shard 1/6. Shard 5/6 carries @objectstack/cli 1/2.
So "spec's own heavy suite only runs on spec PRs" is not what makes shard 5 the outlier;
the slice leg is. Verified by running select-shard-packages.sh + partition-test-shards.mjs
locally over the full list.
Restore Turbo cache reporting success says nothing about a hit: on run 34215028203
the step took 2 s and on 34193080219 it took 56 s, and the longer restore
produced the longer job. Zone 3's distinction is real, and Cached: N cached, M total
in the leg's own summary is the reading that answers it.
noted, not filed: the restore-key chain's last fallback, Linux-turbo-test-, matches
any shard's namespace, which partly undoes the shard scoping the step's comment
describes. Harmless (turbo entries are content-addressed) and not a defect — noted only
because the comment reads as if the scoping were total.

⚠ This PR's own CI run does NOT exercise the change

Measured, not assumed: with TURBO_SCM_BASE at this branch's merge base,
turbo ls --affected returns 0 packages for this diff (turbo.json's
globalDependencies are tsconfig.json and tsup.config.ts — not .github/**).
So every shard on this PR takes the No packages on this shard — nothing to test exit,
the new step short-circuits with it, and a green Test Core here says nothing about the
new code path. That is why the evidence above is offline and local rather than "the PR went
green": the harness drives the real extracted step bodies, and the turbo readings were taken
in a real checkout. The first run that actually executes the new step is one whose affected
set reaches @objectstack/cli — a merge-group run, or the next packages/spec PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU

Generated by Claude Code

…e run, so the shards stop scattering it (objectstack-ai#17867) Clause-②: no Closes objectstack-ai#16886 The `dogfood:` job carried the same run-level Turbo passthrough defect that PR objectstack-ai#16868 fixed in the `test` job for objectstack-ai#16395. It was outside that card's declared surface, so it was still there. This applies objectstack-ai#16868's landed diff as the template rather than a second working spelling — the two jobs are now spelled the same way. ## The site, measured on this branch (⛔ not the card's line number) The card said `.github/workflows/ci.yml:1283`; found by content, the run sits at **`:1520`** on the merge base `c88fa2ccd` and at `:1579` after this diff. The job header is `:1423` and `name:` is `:1431` — the card's `:1186` / `:1209` are 2026-09-08 readings and this file moves constantly. ``` 1520: pnpm turbo run test --filter=@objectstack/dogfood --log-order=stream -- --shard=${{ matrix.shard }}/3 ``` ## The change 1. A dedicated, guard-wrapped, **passthrough-free** `pnpm turbo run build --filter=@objectstack/dogfood --concurrency=4 --log-order=stream` step ahead of the sharded run, so the closure is built once and reaches the shared, content-addressed `build` cache. 2. `--only` on the sharded `turbo run test`, so the passthrough hash covers only the `test` task it is meant for. It is **its own step**, not a second guarded run inside the existing one, for the reason objectstack-ai#16868 states: a guarded SITE is the triple (file, job, step), and `measure-stall-guard-headroom` refuses a verdict when two guarded runs share one. ⛔ `name:` is untouched (required-status-check context) and `timeout-minutes: 30` is untouched — the card names that budget as the instrument that shows the fix working, so it may not be raised to buy room. ## Cache reading — before / after, `--dry=json` task hashes turbo 2.10.10, `--filter=@objectstack/dogfood`, on this branch. The plain test plan is **67 tasks: 66 `build` + 1 `test`**. **Before** — every task in the run re-hashed per shard: ``` plain vs plain (control, fires) tasks 67/67 identical=67 changed=0 plain vs -- --shard=1/3 tasks 67/67 identical=0 changed=67 -- --shard=1/3 vs 2/3 tasks 67/67 identical=0 changed=67 -- --shard=2/3 vs 3/3 tasks 67/67 identical=0 changed=67 ``` So the three shards hashed one 66-package build closure three ways, and none could reach the `build` cache every other job in this workflow populates — that cache is written with no passthrough in the hash. **After** — the two runs the new spelling issues: ``` turbo run build --filter=@objectstack/dogfood 67 tasks, every one a #build task vs the plain test plan's ^build closure (66) hash-identical=66 differing=0 missing=0 (the 67th node is @objectstack/dogfood#build itself, whose command is NONEXISTENT in the dry output because the package declares no `build` script — it executes nothing and is only the root its 66 `^build` dependencies hang from) turbo run test --filter=@objectstack/dogfood --only -- --shard=k/3 k=1: 1 task k=2: 1 task k=3: 1 task -> ['@objectstack/dogfood#test'] --only shard=1/3 vs itself (control, fires) identical=1 changed=0 --only shard=1/3 vs --only shard=2/3 identical=0 changed=1 ``` The 66 closure builds now **replay** instead of re-executing per shard: their hashes are identical to the passthrough-free test plan's, which is what the shared cache holds. The one remaining per-shard hash difference is the `test` task itself, which is correct and intended — that is the task the passthrough is for. ## ⭐ The sweep: `--filter` plus a run-level passthrough, whole tree Triage asked for the count before closing, so here is the probe and the count rather than a claim. **Probe shape.** A tokenizing scan (`shlex`, quote-stripped, backslash continuations joined first) over all 37 files in `.github/workflows/`. For each `turbo run` occurrence it tokenizes **from the `turbo` token onward** and asks three questions: does it carry `--filter`/`-F`; is there a bare `--` token *after* `turbo run` with at least one argument behind it; does it carry `--only`. Starting at the `turbo` token is the load-bearing part — a naive `-- ` grep reports `set -- pnpm turbo run …` (the shell builtin) and `node run-with-stall-guard.mjs … -- pnpm turbo …` (the wrapper's own separator) as passthroughs. That first, naive grep over this same tree produced five hits, of which **three were the wrong `--` entirely**; the classification below is from the tokenizing probe. 37 files, 40 `turbo run` textual matches — **23 executable**, 17 prose in comments (classified, not dropped). **Count: 3 sites carry `--filter` plus a run-level passthrough. 2 of them were the defect; both are now fixed; there is no third.** | site | filter | run-level passthrough | `--only` | verdict | |:---|:---:|:---:|:---:|:---| | `ci.yml:738` job `test`, slice leg | yes | `-- "--shard=$SLICE"` | **yes** | same defect, fixed by objectstack-ai#16868 — the template | | `ci.yml:1577` job `dogfood` | yes | `-- --shard=k/3` | **yes** (this PR) | same defect, fixed here | | `test-nightly-tiers.yml:281` job `tiers`, slice leg | yes | `-- "--shard=$SLICE" $REPORTER` | no | **same defect, outside this card's fence** — filed as objectstack-ai#17866 | **Zeros, reported as asked.** The other 20 executable `turbo run` invocations: 14 carry a `--filter` and **no** passthrough (the plain `turbo run build` legs in `ci.yml`, `lint.yml`, `cut-rc.yml`, `showcase-smoke.yml`); 4 carry neither; and 2 (`rerun-safety-nightly.yml:99`/`:130`) carry neither a filter nor a passthrough. **Zero sites outside the table above carry both.** **One blind spot in the probe, declared.** A filter arriving through a shell variable is invisible to a token-level scan: `test-nightly-tiers.yml:276` is `turbo run test $FILTERS … -- $REPORTER`, where `$FILTERS` is built as `--filter=$PKG` in the loop immediately above it. The probe scores it `filter=0`; read by hand it is a **fourth** instance of the same class, and the only one whose *whole-package* leg carries a passthrough (`ci.yml`'s corresponding leg carries none). Both nightly sites are in objectstack-ai#17866 with their own measurement; they are not fixed here because that workflow is `schedule`-triggered, outside the required set, so a fix would land unexercised by any PR's CI — and the `:276` half needs a derivation the pinned shape does not supply. ## ⭐ Reverse-read: which existing sentence does this make false? **Sentences made false: zero.** The two present-tense claims in range, both in this job: - `Restore Turbo cache`, still on the tree: *"the turbo test hash differs per shard (pass-through args are part of the task hash)"* — still **true**, and still the reason its key is shard-scoped: the `test` task keeps the passthrough. **Left alone.** - Same comment: *"turbo's cache is content-addressed per task, so another shard's entries replay the shared build closure even when the test slice differs"* — this one ran the **other** direction. It was **false before this diff** (the closure was hashed three ways, so another shard's entries could not replay it) and this change makes it **true**. Left alone, now correct. - The test step's *"The `--` args reach the package's `vitest run` and are hashed into the turbo task, so each shard caches independently"* — not false, but no longer the whole story under `--only`. **Extended in place**, not deleted: it now says the passthrough is hashed into that one task and that the closure it used to scatter is built once by the step above. **Sentence kinds, distinguished as asked:** nothing carrying a rev or a date was touched (`objectstack-ai#2018`, `objectstack-ai#4250`, `objectstack-ai#4859`, `objectstack-ai#4928`, the objectstack-ai#16868 measurement block — all history). The named-direction zeros elsewhere in the file (`check:required-contexts` pins, the `!@objectstack/dogfood` exclusion rationale, the "NOT the dogfood job's vitest `--shard` passthrough, deliberately" note at `:295`) are unaffected: the passthrough stays, it only stops being hashed into 66 other tasks. **No bare present-tense magnitude in the tree became stale** — the 66/67 counts this diff introduces are new, and each is written with the tree and the turbo version it was taken against. **Pin tests on the current behaviour: zero found.** `--only` appears in no test or gate assertion about this job; `check:stall-guard-budget` reads the step *structurally* and re-derived it green (below), rather than pinning a step list. ## Gates `scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived **44 commands** from the real change set (1 path, vs merge base `c88fa2ccd`). Ran all 44; reconciled the ran-list back against the deriver. - **39 green.** The ones that read this diff: `check:stall-guard-budget`, `check:stall-guard-headroom`, `check:required-contexts`, `check:agent-test-spelling`, `check-ci-filter-parity`, `check:shard-attestation`, `check-step-collectors`, `check:workflow-step-name-quoting`, `check:workflow-status-functions`, `check:pnpm-filter-targets`, `check:nul-bytes`. - `check:stall-guard-budget` verdict line, which proves the new site was measured rather than skipped: ``` check-stall-guard-budget: OK (37 workflow file(s), 62 job(s), 692 step(s), 10 guard-wrapped step(s); every effective cap clears its job budget by at least one stall window). .github/workflows/ci.yml:1538 job `dogfood` step `Build the dogfood package's dependency closure` window 10m (explicit) · cap 20m (2x window) · budget 30m (job timeout-minutes) · slack 10m (1 of 2 guarded steps in this job; they share one timeout clock) ``` — the same window/cap/budget shape as `test`'s pair, and 10 guard-wrapped steps where the tree had 9. - **4 NOT MEASURED, exit 3 (`PREREQUISITE NOT MET`), not failures:** `check:dts-closure`, `check:dual-build-cjs-loads`, `check:lean-entry-closure`, `check:sourcemap-no-sources-content`. All four read built `dist/` trees and say so in their own refusal text. They are placed by a whole-tree declaration, not by this diff's path, and a workflow-YAML-only diff moves no input of theirs. CI builds and runs them. - **1 NOT MEASURED, incomplete:** `check:pm-dispatch-gates` exceeded a 560-second foreground budget inside its own self-test at 1507 passing assertions and **zero** failures. Left to CI. **Lint scope — a measured narrowing, not a skipped run.** Repo-level `pnpm lint` (`eslint . --no-inline-config`) is CI's run. The narrowing here is that this diff contains **zero files eslint judges**, and all three pieces of evidence: (1) eslint's own answer for the one file, from its own config — `File ignored because no matching configuration was supplied`; (2) `--format json` over the diff — 1 file, **0 errors**, 1 warning, that notice; (3) invariance for untouched files — this repo runs one `eslint.config.mjs` which never enables type-aware linting for any file (`eslint.config.mjs:326-329`, measured there with a positive control), so nothing in this diff can move any untouched file's verdict. Taken at `21d717d5`, the final commit. ## Changeset: skipped, and measured first `skip-changeset` label applied. Nothing published moves: of the **70** packages declaring a `files[]`, **zero** ship any `.github/workflows` path; positive control — `@objectstack/spec`'s `files[]` reads back non-empty with its real shipped paths. The one other `.github` directory in the tree (`packages/create-objectstack/src/templates/blank/.github`) is a different path and is untouched. This is also the fast-track case: a repo-root CI config. ## Clause-② `Clause-②: no`, declared line-initial at the top of this body and self-verified against `readClause2Line()` read fresh from `scripts/pm/check-clause2-carriers.mjs:909` — whose `CLAUSE2_KEY_LINE` now has three capture groups. The substantive answer matches the diff: one CI workflow file, no schema key, no closed-set member, no published export, no registry entry, so no C5 widening tell either. --- _Generated by [Claude Code](https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU)_ Co-authored-by: Claude <noreply@anthropic.com>

claude added 2 commits September 8, 2026 11:48

wip(ci): slice leg builds its closure in a passthrough-free run

f03b6cc

wip(ci): slice closure builds in its own guarded step

afbddd7

claude Bot added domain:devx skip-changeset PR has no user-facing published change; bypasses the changeset gate tooling labels Sep 8, 2026

github-actions Bot added size/s ci/cd labels Sep 8, 2026

baozhoutao marked this pull request as ready for review September 8, 2026 13:26

baozhoutao enabled auto-merge September 8, 2026 13:27

baozhoutao added this pull request to the merge queue Sep 8, 2026

Merged via the queue into main with commit d958b34 Sep 8, 2026
52 checks passed

baozhoutao deleted the claude/issue-16395-spec-shard-timeout branch September 8, 2026 13:48

baozhoutao mentioned this pull request Sep 12, 2026

CI: test-nightly-tiers.yml carries the same run-level Turbo passthrough defect — both legs, and its whole-package leg has it too #17866

Closed

claude Bot mentioned this pull request Sep 12, 2026

ci(dogfood): the sharded gate builds its closure in a passthrough-free run, so the shards stop scattering it #17867

Merged

Navigation Menu

Repository navigation

ci(test): a file-level slice builds its dependency closure in a passthrough-free run, so the sharded leg stops rebuilding it - #16868

ci(test): a file-level slice builds its dependency closure in a passthrough-free run, so the sharded leg stops rebuilding it#16868
baozhoutao merged 2 commits into
mainfrom
claude/issue-16395-spec-shard-timeout

claude Bot commented Sep 8, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

claude Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Root cause, measured — the rebuild is real, and one half of it is removable

The change

How I convinced myself this is correct beyond "the gates passed"

Re-taken timing on current main (the card's numbers are from 2026-09-06)

Acceptance notes

⚠ This PR's own CI run does NOT exercise the change

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

claude Bot commented Sep 8, 2026 •

edited

Loading

Re-taken timing on current `main` (the card's numbers are from 2026-09-06)