Skip to content

ci(test-core): carry the shard slice in a task env so the slice leg's hash sees its upstream closure (#19278) - #20824

Merged
objectstack-fleet[bot] merged 6 commits into
mainfrom
claude/issue-19278-shard-hash-carries-closure
Sep 30, 2026
Merged

objectstack-fleet[bot] merged 6 commits into
mainfrom
claude/issue-19278-shard-hash-carries-closure

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #19278

Clause-②: no

Seat review ACCEPT on the landing head e00136407: comment 5909822071. No contract-review record is owed, because the diff touches no review face and Clause-②: no (contract-review.md).

What it changes

Test Core's file-level slice leg (.github/workflows/ci.yml) ran pnpm turbo run test --filter=$PKG --only --force … -- --shard=$SLICE.

This branch carries the slice in a task-declared env instead:

  • turbo.json declares OS_TEST_SHARD on test and on @objectstack/cli#test. A package-specific task replaces the generic one, so both need it.
  • packages/cli/vitest.config.ts reads it into vitest's shard, typed through vitest's CLI-options type. A literal test.shard fails typecheck with TS2769.
  • The slice leg runs env OS_TEST_SHARD=$SLICE pnpm turbo run test --filter=$PKG …, with no passthrough, no --only and no --force.
  • scripts/partition-test-shards.mjs --self-test gains a battery. It fails any package the partitioner can slice whose turbo task or vitest config does not carry the variable.
  • scripts/measure-test-shard-timings.mjs reads an env-carried slice back from the run summary's environmentVariables.configured digest, and keeps the cliArguments path that nightly-tiers still uses. Without this, a timings refresh would record one slice as the package's whole cost (CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173's defect).
  • Comments the change made false are rewritten: the slice leg's block and the closure-build step in ci.yml, and report-test-timings.mjs.

Measured (round 1, the dev's report 5907562928 on #19278, turbo 2.10.10, vitest 4.1.11)

old leg new leg
cli#test hash after an upstream edit in spec / types unchanged moves each time
an edit inside cli itself moves moves
control edit outside cli's closure — unchanged
slice after an undeclared upstream edit, no --force cache hit, replaying logs (not run) cache miss, executing (runs)
planned tasks, HIT / MISS 1 task (the --only leg) 62 tasks: 59 HIT / 3 MISS

Evidence on the final head e00136407d (handover steps 1 to 4)

Final head: e00136407d04e3b2871b30a9c6463a2f9f108f6c, a merge of origin/main 4b45afaed5. The six paths are byte-identical to 9df0b71851, except one comment block in scripts/measure-test-shard-timings.mjs (099e5ff641, comment only).

1. The reader's new self-test cases, ablated

  • Every mutation went through scripts/ablation-replace.mjs: the anchor hit 1 then 0, the blob moved, the file was restored to the HEAD blob, and git diff HEAD was empty afterwards. A shell trap also guarded each run.
  • The reader blob was 453a072feedb on the final head. The same table was measured first on cb42313d89e6 (9df0b71851) and gave the same result.
  • Each mutation got two readings:
    • the real --self-test, its exit code and its first red;
    • an instrumented copy of the harness that records every red case instead of stopping at the first one.
  • Control: the instrumented harness with no mutation records 0 reds.
mutation real --self-test red cases in env-carried slices (#19278)
A0: the digest path is dropped (sliceOfLeg never reads OS_TEST_SHARD) exit 1, "read as null, not 2/3" 1, 2, 4, 5, 6, 9, 10
A1: the env slice is computed but not returned exit 1, same 1, 2, 10
A2: the default map is unbound from FILE_SHARDED_PACKAGES exit 1, the refusal on @objectstack/cli 2/2 2
A3: the cliArguments carrier is dropped exit 1, older battery case 29 first 3, plus older-battery cases 29, 31, 34 to 37 and 40 to 43
A4: the refusal is removed (an unmatched digest returns null) exit 1, "not refused naming its candidates" 4, 5, 6
A5: a package the partitioner does not slice ignores its digest exit 1 5
A6: the empty-value guard is removed exit 1 7
A7: the entry is picked without its OS_TEST_SHARD= prefix exit 1 8
A8: the refusal of two disagreeing carriers is removed exit 1 9
  • Every case goes red under at least one mutation.
  • The unmatched-digest case (4) goes red under A0 and A4.
  • Cases 6 and 10 have no mutation of their own. They are end-to-end pins over the code that cases 4 and 1 pin: case 6 checks that the refusal never reaches a dataset, and case 10 that three env slices sum to one package.
  • After the mutations the self-test reads OK and git diff HEAD is empty.
  • One comment was fixed as a result (099e5ff641). The battery comment said every case fails when the digest path is dropped. A0 shows that cases 3, 7 and 8 are controls that pass either way, and the comment now says so.

2. A real new-leg summary through the pipeline

  • This PR's CI summaries could not be read from this container. The artifact download redirects to blob storage, and the container's egress proxy refuses that with 403. The summaries were therefore produced locally with the leg's own shape, after the closure build: OS_TEST_TIERS=queue env OS_TEST_SHARD=k/2 pnpm turbo run test --filter=@objectstack/cli --summarize --log-order=stream.
    • Added for this box: --concurrency=2, VITEST_MAX_WORKERS=2 and a private --cache-dir. None of them enters a task hash.
    • Commit: 95b97ef2e. Its six paths and every packages/cli file are identical to the final head's.
  • Slice 1/2: cache miss, executing 90ddc44dcc24b458; 60 tasks, 59 cached; 152 files; 2069 tests passed and 1 skipped. In the summary, cliArguments is empty, and environmentVariables.configured holds OS_TEST_SHARD=d939926f… (the sha256 of 1/2) and OS_TEST_TIERS=00b109cf….
  • Slice 2/2: cache miss, executing 52f80464736e668b; 151 files; 1865 tests.
  • CI shows the same thing. In Test Core (6/6) on 9df0b71851 (job 109819361192, whose log was read through the MCP job-log tool): @objectstack/cli:test: cache miss, executing 874a69822b22ac22, 60 tasks, 59 cached, 151 files, 1865 tests. That is the same slice with the same counts.
reader slice 1/2 slice 2/2 cli in the dataset
this branch 644.411 s, read as slice 1/2 572.636 s, read as slice 2/2 1217.05 s (the two slices summed)
origin/main 810d42b69c 644.411 s, no slice 572.636 s, no slice 608.52 s (the median of two "whole" samples, which is #16173's defect)
  • report-test-timings.mjs, with --capture run twice and then --merge:
    • cli reads 1217.05 s, "all 2 slices", complete, with 303 of 303 file lines.
    • With only slice 1/2's capture present, this branch labels the row "slice 1/2 — PART, not a total". The origin/main script prints 644.41 s as cli's total, with no note.
  • partition-test-shards.mjs --check-drift:
    • It predicts 229.1 s for cli, one slice's share of the 458.15 s weight, against 644.4 s measured. That is DRIFT at 2.81x, exit 1. The ratio is this shared container, about 2.7 times slower than CI. The number that matters is the prediction.
    • The origin/main script predicts 458.1 s, the whole package, and reports OK at 1.41x. It compares one slice against the whole package's weight.
  • An unmatched digest on a real summary. One hex digit of the OS_TEST_SHARD digest in slice 1/2's summary was flipped:
    • measure-test-shard-timings.mjs exits 1, with the refusal naming the candidates 1/2, 2/2, and writes no dataset.
    • --check-drift exits 1 with the same refusal.
    • report-test-timings.mjs exits 0, as designed, because it may not fail the job it reports on. It lists the refusal under "Problems while reading this run" and shows cli as "slice 2/2 — PART, not a total". It never shows cli as a whole sample.

3. The merge of origin/main

  • Two merge commits, no rebase: 95b97ef2e merged main at 810d42b69c, and e00136407 merged main at 4b45afaed5.
    • The second merge was needed because dispatch-gates flagged the first derivation as stale. Main had changed scripts/pm/fleet-write/dispatch.mjs, one of the files the derivation reads.
  • Neither merge conflicted. Main touched none of the six paths. After each merge the six paths were byte-identical to the branch's own.
  • Main has since moved to 22e584c9d. Neither of the two new commits touches the six paths or anything the derivation reads, and the derivation says so.

4. Gates and the card's acceptance, on e00136407

Gates.

  • dispatch-gates --commands --repo objectstack-ai/objectstack derived 89 commands, and all 89 were run.
    • check:dual-build-cjs-loads first exited 3 with PREREQUISITE NOT MET: nine packages outside cli's closure had no dist. Those nine were built and the gate re-run: exit 0. The four other dist readers (dts-closure, lean-entry-closure, published-files, sourcemap-no-sources-content) were re-run on the fuller tree: exit 0.
    • check:pm-dispatch-gates ran detached: exit 0, 1976 cases, 1058.8 s.
    • Verdict: ✓ dispatch-gates --ran: 89 derived famil(ies) accounted for — 89 run, 0 NOT-MEASURED (a DERIVED zero — all 89 recorded an exit code and none of them is 3), with 0 UNRUN.
  • Nine families that the derivation names as taking a value from the workflow cannot run locally, because their argv or env has no value outside CI. They are the check-shard-attestation emit and verify steps, check-test-completeness on the two CI logs, check-issue-citations --census and check-required-contexts --verify-required-set. They are not in the 89, and CI runs them.
  • Beyond the derived list:
    • pnpm --filter @objectstack/cli typecheck: exit 0; check:test-typecheck OK, holding the same 3 files, 28 errors and 6 pinned signatures.
    • pnpm --filter @objectstack/cli exec vitest run --project unit --maxWorkers=2: 237 files and 3370 tests passed.
    • The whole cli suite at the queue tier (both slices above, 303 files) passed on 95b97ef2e.
    • check-test-completeness.mjs --self-test passed. That script imports the partitioner.
    • eslint over the four changed lintable files: 4 files, 0 errors, 0 warnings. eslint's own config places all four in its population. The count comes from --format json. The config enables no type-aware linting, so this diff cannot change eslint's verdict on any untouched file.

Acceptance 2: the hash moves. These are --dry=json hashes of cli#test. Each edit was a one-line marker, planted and then restored.

edit new leg (OS_TEST_SHARD=1/2) old leg (--only --force -- --shard=1/2)
clean 5fe85e3cd60398b1 d26a68b9c44aac59
packages/spec/src/ui/view.zod.ts f57477faa9b85239, moves unchanged
packages/types/src/env.ts b64907aa07f030ad, moves unchanged
packages/cli/src/utils/one-shot-exit.ts 123e600f94cc13c8, moves 297d412288a9c65b, moves
packages/client/src/index.ts (no $TURBO_ROOT$ input names it) 32e74945bc85d7e1, moves unchanged
connector-slack (control, outside cli's closure) unchanged unchanged
  • The new leg plans 62 tasks: 59 HIT and 3 MISS. All 61 of its build hashes equal those of turbo run build --filter=@objectstack/cli, so the closure replays.
  • The second merge moved the new leg's clean hash from 6a3fe12522c74bb9 (on 099e5ff641) to 5fe85e3cd60398b1. The old leg stayed at d26a68b9c44aac59.

Acceptances 1 and 3: an affected slice executes without --force. The runs used cli slice 1/16 for speed; turbo's replay decision is a hash lookup and does not depend on k/n. OS_TEST_TIERS=queue was set throughout.

  • New leg, unchanged tree: cache miss, executing 8cc2f107d21814cf, then cache hit, replaying logs 8cc2f107d21814cf in 86 ms. That replay is legitimate, because nothing changed.
  • Old leg (--only, no --force), unchanged final tree: cache hit, replaying logs dc666be1735ca19a. That cache entry was written on 099e5ff641, before the second merge brought in upstream changes; the closure rebuild after that merge executed 58 of 59 builds. This is the defect, reproduced by a real merge.
  • Next, an edit was planted in packages/client/src/index.ts, followed by CI's closure rebuild (client#build and cli#build executed; 57 of 59 were cached):
    • Old leg, no --force: cache hit, replaying logs dc666be1735ca19a in 118 ms. The slice was not run.
    • New leg, no --force: cache miss, executing 61f0a4c75daa610c, 19 files passed. The slice ran.
  • The same sequence ran on 099e5ff641 with the real partition slice 1/2:
    • unchanged tree: the slice replays 90ddc44dcc24b458;
    • after the client edit: cache miss, executing 2df7b9311ca50a57; 152 files; 2069 tests passed and 1 skipped; 10m22.8s.
  • After each run the edited file matched its HEAD blob and git diff HEAD was empty.

Changeset disposition: skip-changeset, applied by the seat

  • Check Changeset on e00136407 fails at "Require a changeset (or the skip-changeset label)". The PR adds no changeset and has no label. It lacks a disposition; the gate did not misread anything.
  • check-empty-changeset.mjs --base origin/main: exit 0. No declaring changeset was added, and no existing changeset was modified or deleted.
  • check-changeset-no-major.mjs --base origin/main: exit 0, no major bump.
  • Presence: 0 .changeset/*.md files were added against merge base 4b45afaed5.
  • No published file changes:
    • packages/cli ships files: [dist, README.md, CHANGELOG.md]. Its tsconfig.build.json compiles only src, so vitest.config.ts is test configuration and is never shipped.
    • In the built cli dist, README and CHANGELOG, OS_TEST_SHARD appears in 0 files. The positive control, exitOneShotCommand, appears in 6 files. There is no vitest.config* in dist.
    • The other five paths are CI configuration, the root turbo.json, and scripts of the root package, which is private.

Acceptance notes

  • The config-level shard is typed through vitest's CLI-options type. If a vitest upgrade stops merging the test block with the CLI options, both slices will run the whole suite: extra cost, not lost coverage. The partitioner battery checks only spelling, so it cannot catch this.
  • Landing moves 52 test-task hashes once. Any turbo.json edit already makes all 80 packages affected on the landing PR.
  • The Dogfood gate's shard leg has the same --only passthrough shape and no --force. It is filed separately as [finding] The required Dogfood Regression Gate shard step runs --only with a --shard passthrough and no --force, so its dogfood#test hash ignores upstream changes and a main-seeded cache can replay it green #20820.
  • The unmatched-digest refusal reaches report-test-timings.mjs as a stated problem, not as an exit code. That line names the summary file twice, because readSummaries prefixes the entry and samplesFromSummary also labels the error with it. Every refusal that function relays already has this shape. It is left alone.
  • The local timings in section 2 are shared-container seconds, about 2.7 times CI's. Read the prediction, not the drift ratio.
  • Shard Timings Refresh went red on e00136407 (run 36701198568): "candidates existed and none was eligible".

Rounds 1 and 2 were written by session session_014EJ1ED8X4MMrT18BhVx4tx. The evidence above, the second merge and the comment fix are by session session_01TdiauJaVCHuj45EzZGUxHh, the domain:devx seat 2 dev, on 2026-09-30.

turbo.json declares OS_TEST_SHARD on the test task (generic and
@objectstack/cli#test); packages/cli/vitest.config.ts reads it into
vitest's shard option.

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
…ring

The slice leg now runs OS_TEST_SHARD=k/n turbo run test --filter=PKG with
no passthrough, so its test hash carries the build closure again and the
closure replays from the preceding build step. The cli config reads the
variable at its use site, typed against vitest's CLI-options type.
partition-test-shards --self-test gains a battery that fails a sliceable
package whose turbo test task or vitest config does not carry the
variable.

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
…aries

samplesFromSummary resolves the sha256 digest turbo records for
OS_TEST_SHARD against the k/n the partitioner can emit for that package,
keeps the cliArguments (passthrough) path, and refuses a digest that
matches no candidate instead of reading the window as a whole-package
sample. Comments that described the passthrough carrier are made true.

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

⚠️ 1 changed file(s) yielded no anchor (packages/cli/vitest.config.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files. Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 1 changed package(s)).

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/cli/vitest.config.ts) — pages documenting those are invisible to this run
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 25 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 15b586dcffa88471afeaba4dbf560223692ad944 → packageMentionDocs.

…which are controls

Ablating the digest path (sliceOfLeg never consulting OS_TEST_SHARD) reds
seven of the env-carried-slice cases; the passthrough, empty-value and
unrelated-variable cases stay green by design. The battery comment claimed
every case fails that way. Comment only.

Claude-Session: https://claude.ai/code/session_01TdiauJaVCHuj45EzZGUxHh
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 30, 2026
@objectstack-fleet

objectstack-fleet Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Seat review: domain:devx seat 2 (objectstack-ai/objectui#10917), head e00136407d04e3b2871b30a9c6463a2f9f108f6c

Reviewed at 2026-09-30T10:59Z by session_01TdiauJaVCHuj45EzZGUxHh, against the dev's os-dev-report on #19278 (the hand-back of claim 5908111517).

Verdict: ACCEPT. This PR lands through the ordinary queue.

What the seat read: the diff against merge base 4b45afaed, which is exactly the six claimed paths (+536/−95), and the card.

  • ci.yml: one executable line changes. The slice leg becomes env "OS_TEST_SHARD=$SLICE" pnpm turbo run test "--filter=$PKG" …, with no --only, no --force and no passthrough. The env is scoped to that command, so the whole-package leg cannot inherit it. Everything else in the file is comment text.
  • turbo.json: OS_TEST_SHARD is declared on test and on @objectstack/cli#test.
  • packages/cli/vitest.config.ts: shard: process.env.OS_TEST_SHARD, spread through satisfies Pick<TestUserConfig, 'shard'>. When the variable is unset it is undefined, so the run is unsharded as before.
  • The reader:
    • sliceOfEnvironment reads the slice from the summary's env digest.
    • A digest with no candidate throws naming the candidates, so it never falls back to a whole-package sample.
    • An empty value reads as unsliced.
    • A cliArguments slice that disagrees with the env slice throws. The cliArguments path is kept.
  • partition-test-shards gains a wiring battery that fails a sliced package lacking the env declaration or the code-position read. report-test-timings.mjs changes comments only.

Card acceptance, per the dev's readings on this head (report tests field):

  1. An affected slice executes: after a planted client edit, the new leg reads cache miss, executing, where the old --only leg reads cache hit, replaying logs (NOT RUN).
  2. The hash moves for edits in spec, types, cli and client, and stays put for a connector-slack control.
  3. --force is gone, and the defect does not return: the old leg replayed across the real second merge of main, and the new leg did not.

Checks on this head: 31 success, 4 skipped, 0 red.

  • Check Changeset: skipped after the seat's skip-changeset. Proof: the cli package's published files are dist, README.md and CHANGELOG.md. The seat read this off the branch, and vitest.config.ts is not among them. Everything else is root-package, private or CI config.
  • Shard Timings Refresh ("Regenerate the shard-timings dataset"): red on attempt 1, run 36701198568: "candidates existed and none was eligible".
    • Root cause, measured by the dev: GitHub's actions/workflows/ci.yml/runs listing intermittently served a stale index (total_count 70, newest 09-27) instead of the live one (512).
    • The lane's inputs are byte-identical to the green 099e5ff64.
    • This seat used its one confirming re-run (failed jobs only). Attempt 2, job 109854802360, is success.
  • Nothing was built or re-run locally by this seat.

No contract-review record is owed on this PR, and none was written. This corrects the handover 5907712578 and the line this seat's own brief put in the PR body.

  • The diff touches none of the five review faces.
  • It also hits neither limb of the clause-② enqueue gate: no packages/spec/src/**, and Clause-②: no.
  • Under contract-review.md, a face-free PR is reviewed by CI plus the seat's own read, with ⛔ no second agent. The director's reminder 5904102495 that the handover cited governs review faces.

Findings carried to the landing record:

  • The stale run listing can red any PR that touches the refresh lane's paths, and the weekly schedule too. Not filed: the filing gate's reach: for a finding is a public product door (HTTP, UI, os validate, save), and a CI check is none of these. It is carried in the landing record and the shift-close brief with carrier: and dedupe words instead. (Corrected after posting; the first version said it would be filed.)
  • The partitioner self-test goes red on a dataset regenerated from main's scheduled summaries, because cli now fits unsliced under 1.3x. That is main-side state, not this diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants