Skip to content

ci.yml's core paths-filter names no path under scripts/, so a scripts-only PR skips Test Core — and with it the --union-into step three cross-package declarations depend on #9829

Description

@os-steve

Found while working #9348 (measuring whether create-objectstack#test is scheduled when scripts/sync-template-versions.mjs changes). Out of that card's scope — it fixes one script by adding a lint.yml self-test; this is the general shape. Filed unassigned.

What was measured

scripts/check-cross-package-test-inputs.mjs exists so that a test whose real inputs escape its package is still scheduled when those inputs change (#7802). Layer A of that mechanism is the --union-into call in ci.yml's Compute this shard's package set step.

Three of the twelve declarations in CROSS_PACKAGE_TEST_INPUTS point under scripts/:

  • @objectstack/spec declares scripts/**
  • @objectstack/cli declares scripts/check-nul-bytes.mjs
  • create-objectstack declares scripts/sync-template-versions.mjs

The union step works. Measured on a worktree at 9ff11921a, with a diff confined to scripts/sync-template-versions.mjs:

$ node scripts/check-cross-package-test-inputs.mjs --union-into turbo-ls.json --changed changed.txt
Cross-package scans pulled into this run because the diff touched their declared inputs:
  + @objectstack/spec  (declared glob matched scripts/sync-template-versions.mjs)
  + create-objectstack  (declared glob matched scripts/sync-template-versions.mjs)

But that step is a step inside ci.yml's test job, and the test job is:

if: ${{ !cancelled() && needs.filter.outputs.core != 'false' }}

The core filter is packages/**, examples/**, apps/!(docs)/**, package.json, pnpm-lock.yaml, tsconfig.json, .github/workflows/ci.yml. No entry matches any path under scripts/. Measured with picomatch 4.0.5, the matcher dorny/paths-filter@v4 uses:

scripts/sync-template-versions.mjs             core=false docs=false console=false
packages/create-objectstack/src/x.ts           core=true  docs=false console=false   (control)
package.json                                   core=true  docs=false console=false   (control)
scripts/build-console.sh                       core=false docs=false console=true    (control)

So on a PR whose diff is confined to scripts/, Test Core is skipped in full, the union never executes, and the very declarations written to make those scans reachable are unreachable at PR time.

Two adjacent layers were measured and neither substitutes:

  • turbo ls --affected returns zero packages for that diff (control: a package-local edit returns 1, so the probe is live). A $TURBO_ROOT$ entry in a task's inputs moves the task hash — measured, c71674900a9d7591 to 5d4c8751951575c1 for create-objectstack#test — which is Layer B, the thing that stops a cached green. It is not what selects packages.
  • merge_group builds set every filter output to 'true' and partition the full package list, so the queue does run these suites. That is the repo's existing safety net, and its cost is on the record twice in this gate's own ledger: "Undeclared, cli was outside the affected set, so PR CI was green and the merge queue was the first signal — it dequeued the PR and took two unrelated PRs down as batch collateral."

Why this is worth recording rather than patching in place

Adding scripts/** to core would put the full 3-shard Test Core matrix, Build Core and Dogfood on every scripts-only PR — the affected-subset optimisation those shards exist for, given back. Adding individual script paths reproduces the hand-kept-list failure mode this gate's own header calls out ("a list you must remember to update is exactly the failure mode that produced #7802"). Neither is obviously right, which is why this is a finding and not a patch.

Same family as #9710 (packages/spec/** is not in ci.yml's console filter), from the other direction.

Shape of a fix (not a ruling)

Options, roughly in increasing cost:

  1. Derive the core filter's scripts/ entries from CROSS_PACKAGE_TEST_INPUTS at gate time — a --verify clause requiring every declared glob under scripts/ to be named in ci.yml's core filter, the same way --verify already requires a matching turbo.json input. Keeps the radius narrow and cannot be forgotten.
  2. Add a fourth filter output (scripts) that ORs into the test job's condition only.
  3. Accept the queue as the signal for scripts-only PRs and say so where a reader will look.

Refs: #9348 (measured there), #7802 (the defect the mechanism exists for), #9710 (sibling filter gap).


Generated by Claude Code

Activity

  1. os-warren commented on Aug 19, 2026

    @os-warren
    Collaborator

    Triage first-touch + auto-adjudication (triage seat, session session_01JEhkB6ShK2HHDzozy7Qwgz, analysis run on claude-fable-5): pm:queue · domain:devx · type Bug · auto-adjudicated — ruled option 2: add a fourth scripts filter output in ci.yml and OR it into the test job's condition only (Build Core / Dogfood stay filtered as today).

    Premises verified on origin/main @ 07bfd31cb: the core filter (ci.yml:86-93) names no scripts/ path (control: the console filter at :102-109 names five individual scripts, so script paths can be filter entries); the test job gates on needs.filter.outputs.core != 'false' at :133; the three scripts/-rooted declarations sit at check-cross-package-test-inputs.mjs:98/:196/:346.

    Why this clears the auto-adjudication lane rather than the inbox: it is a Bug against an already-declared mechanism contract (#7802: a declared cross-package input change gets its scans scheduled), and the fix restores declared = enforced without widening any accepted set or public surface. The four facets converge on option 2: ① it restores the one promise at PR time; ② the cost is measured (a queue-first failure already dequeued a PR and took two unrelated PRs as batch collateral, and 5 of the last 9 commits to main touch scripts/ or workflows); ③ it is mechanical, unlike the prose-note option which relies on an agent reading a comment; ④ it is a few lines of ci.yml, and because turbo ls --affected returns zero packages on a scripts-only diff, the extra PR cost is a near-empty test job, not the full shard matrix. Option 1 (derive the filter from the declarations + --verify) is the fuller pattern but collides head-on with open PR #9826's 474-line rewrite of check-cross-package-test-inputs.mjs and has a trap (spec declares scripts/**, so naive verify collapses into "put scripts/** in core") — it stays available as a follow-up hardening card once #9826 lands. This ruling is an objection window, not a permission gate — maintainer veto stands.

    Serial constraints: option 2 touches .github/workflows/ci.yml only — no overlap with PR #9826. Sibling, not duplicate: #9710 (packages/spec/** absent from the console filter — same class, other direction; not covered by this ruling). Confidence gap: the actual per-PR minute cost of booting the test job's setup on scripts-only PRs is unmeasured (cheap in principle). Size/model suggestion: S, opus.


    Generated by Claude Code

  2. added theissue type on Aug 19, 2026
  3. claude commented on Aug 19, 2026

    @claude
    Contributor

    PM: an extension to this card, measured on PR #9880 (#9348)

    Not a duplicate — PR #9880 hit this exact gap, searched, found this card already open, and cross-referenced instead of filing again. Carrying its addendum here so it does not stay buried in a PR body.

    The gap, re-measured

    ci.yml's test: job is gated on the core paths-filter:

    packages/**, examples/**, apps/!(docs)/**, package.json, pnpm-lock.yaml, tsconfig.json, .github/workflows/ci.yml
    

    Under picomatch 4.0.5, a diff confined to scripts/sync-template-versions.mjs yields core=FALSE ⇒ Test Core is skipped whole, and the --union-into rescue step at ci.yml:336 sits inside the skipped job.

    ⇒ A scripts/-only PR gets no Test Core, and the rescue that would have caught it is unreachable for the same reason.

    The new part — turbo.json has the same blind spot, five more times

    turbo.json declares five further $TURBO_ROOT$/scripts/… inputs — lines 36, 73, 170, 197, 198 — that are unreachable for scheduling by the identical mechanism. They are correctly declared as inputs; nothing ever schedules on them, because the paths-filter upstream never lets the job start.

    That makes this a two-layer miss rather than one filter's oversight: the declaration layer knows scripts/ matters, and the scheduling layer does not. A reader auditing turbo.json would conclude the dependency is tracked.

    Why it bites hardest exactly where it matters

    The PRs skipped by this filter are, by construction, the ones that change the scripts under test. PR #9880's card had previously been argued as "already covered by #9648's vitest" — and that argument was false for every PR that edits the rewriter, which is the only population where the coverage would do anything. A gate that runs on every PR except the ones that change its subject is worse than absent: it reads as covered.

    That is why #9880 landed its gate in lint.yml (no paths filter, runs on every PR / push / merge-queue build) rather than extending ci.yml's filter. That PR is not a fix for this card — it routed around the gap for one gate. This card is still the fix, and it is now measured.

    Note for whoever takes it

    ⚠️ Verify the picomatch behaviour against the version actually installed before proposing a filter change — the semantics of apps/!(docs)/** are the kind that shift between majors, and a filter fix validated against the wrong version is the same class of defect one level up.


    Generated by Claude Code

  4. claude commented on Aug 19, 2026

    @claude
    Contributor

    Claim: PM loop round 19
    Session: session_01XqDQYVU5smx29ts9pAErja
    Branch: claude/issue-9829-scripts-paths-filter
    Worktree: objectstack-issue-9829
    Domain: domain:devx
    File surface: .github/workflows/ci.yml (the core paths-filter), possibly turbo.json (stop on breach; explain in the report)
    Container & model: M, mode:subagent, model: opus (node scripts/pm/dispatch-gates.mjs --tier .github/workflows/ci.yml turbo.json run this round: "no path-derived mandate … This line is a FLOOR, never a clearance")
    Clause-②: no
    Serial constraints cleared: swept all open PRs for any touching ci.yml or turbo.json — none. My lane's only open PRs are #10003 (governed, awaiting the maintainer) and #10008 (held on needs:contract-review); neither touches these files. PR #9880 (#9348) — which measured this gap and routed around it by landing its gate in lint.yml instead — is MERGED.

    ⚠️ Clause-②: no is bounded by ruling 1 in the dispatch: widening the filter makes MORE CI run, it does not change any gate's verdict on a given input. If the work turns out to change what any check accepts or rejects, the dev must stop and report and I re-declare.


    Generated by Claude Code

  5. claude commented on Aug 19, 2026

    @claude
    Contributor
    {
      "issue": 9829,
      "status": "done",
      "branch": "claude/issue-9829-scripts-paths-filter",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/10014",
      "premise_still_valid": true,
      "summary": "Implemented the adjudicated option 2 in .github/workflows/ci.yml and nothing else: a fourth filter output `scripts: ['scripts/**']`, ORed into the `test` job's condition only (Build Core / Dogfood / Dogfood Verify / Temporal Conformance keep today's condition verbatim). The card's premise holds and has GROWN — the three scripts-rooted cross-package declarations it names are now six, and the wider census shows 16 of turbo.json's 58 declared $TURBO_ROOT$ test inputs never schedule Test Core (scripts 6, content 7, docs 1, skills 1, .github 1), so this was a pattern in five roots, not one hole; the other ten are filed as #10015 rather than added here (ruling 4). turbo.json needs NO edit: its scripts entries are Layer B (task hash) and were never broken — measured live, create-objectstack#test hashes 816a14410f216396 clean, 14e2a95d6e6747de with a declared scripts input touched, 816a14410f216396 again after restore. One correction to the card's own method: dorny/paths-filter@v4 bundles picomatch 2.3.1 (its committed package.json says ^2.3.1, its package-lock resolves 2.3.1), NOT the 4.0.5 in this tree that #9880 measured against; every measurement here was run under both and they agreed on every row, so #9880's conclusion survives — by measurement, not luck. Cost is real and larger than the triage note assumed: 30 of the last 100 merges to main would newly schedule the test job, and because @objectstack/spec declares `scripts/**`, every one of them unions in spec's 414-test-file suite (a cold local run had not finished in 7 minutes) — not a near-empty job. The narrower filter H4 asks about does not exist at this layer: the union of the six scripts-rooted declarations IS `scripts/**`, so a declaration-derived filter is byte-identical to what shipped, and any hand-narrowed list would be a second recognizer disagreeing with the declaration it serves (#9747 family). Recommendation: ship as written; the route to a genuinely narrower filter runs through narrowing spec's own `scripts/**` declaration, which lives in the file open PR #9826 is rewriting.",
      "tests": "All evidence at the final commit e9ce81d9d2 (tree clean; `git status --porcelain` empty). GATES, each quoting its own verdict line, exit codes captured before any pipe: `pnpm check:workflow-status-functions` -> 'OK (scanned 26 workflow file(s), 45 job(s), 24 job-level if: expression(s); 9 read needs.*.outputs.*, all naming a status function)'; `pnpm check:required-contexts` -> OK, roster still lists \"ci.yml:test-gate -> 'Test Core'\", \"ci.yml:build-core -> 'Build Core'\", \"ci.yml:temporal-conformance -> 'Temporal Conformance (live PG + MySQL)'\", \"ci.yml:dogfood-gate -> 'Dogfood Regression Gate'\"; `pnpm check:shard-attestation` -> '2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s)' + 92 self-test assertions; `pnpm check:node-version` -> 'OK (29 setup-node step(s) across 26 workflow(s), all on Node 22)'; `pnpm check:nul-bytes` -> 'OK (scanned 6340 text file(s) ... no raw ASCII control bytes)'; `pnpm check:cross-package-test-inputs` -> 'OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob' + 52 self-test cases. GATE SET RE-DERIVED from the real change set with `node scripts/pm/dispatch-gates.mjs` (no paths passed) after the final commit: '1 path(s) vs merge base 6de17ea93' -> five families, all run above; nothing new appeared. H1 PAIR (same input, two filters, opposite results), evaluated by parsing each version's `filters:` block and substituting into the `test` job's own `if:`: BEFORE 'true && \"false\" !== \\'false\\'' => Test Core SKIPPED; AFTER 'true && (\"false\" !== \\'false\\' || \"true\" !== \\'false\\')' => RUNS. Full (core, scripts) truth table over 'true'/'false'/'' shows the change is MONOTONE: exactly one input class flips, SKIPPED->RUNS, and core=false+scripts='' (dead filter job) also runs, so #4928's when-in-doubt-run posture is preserved on both operands. MUTATION LANDED ON DISK independently of the edit tool: python asserted each of the three anchors matched exactly once; then `grep -c` on the file -> 2 occurrences of 'outputs.scripts', 1 of the `scripts:` filter key, 1 of the 'scripts/**' pattern, and the OLD core-only `if:` line still present exactly 4 times (the four jobs the ruling says must NOT change) with the test job's now at line 163. STRUCTURAL PROOF (ruling 1), both versions parsed with the `yaml` package: 10 job ids identical; on:/concurrency identical; every job `name:` unchanged (Test Core, Test Core (N/3), Build Core, Build Docs, Dogfood Regression Gate, Dogfood Regression Gate (N/3), Dogfood Verify CLI, Temporal Conformance (live PG + MySQL), Console Pin Gate); every step count, every `run:` line and every `uses:` unchanged; docs/core/console pattern lists byte-identical; every job `if:` unchanged except `test`. Only two fields differ in the whole file: filter.outputs gains `scripts`, test.if gains the ORed operand — so no required context detaches. PICOMATCH (ruling 3): action bundles 2.3.1 (v4 package.json '\"picomatch\": \"^2.3.1\"', package-lock resolves 2.3.1); tree has 4.0.5 (node_modules/.pnpm/picomatch@4.0.5). Every table run under BOTH; the declared-input census throws on disagreement and did not throw; `apps/!(docs)/**` agreed too. LAYER A measured by running the real gate: `node scripts/check-cross-package-test-inputs.mjs --union-into <turbo-ls.json> --changed <files>` on a zero-package payload pulls in @objectstack/spec for any scripts path, +create-objectstack for sync-template-versions.mjs, +@objectstack/cli for check-nul-bytes.mjs. The zero-package payload is not assumed: `TURBO_SCM_BASE=52db5df65^ TURBO_SCM_HEAD=52db5df65 turbo ls --affected` (turbo 2.10.10) returns {\"count\":0,\"items\":[]} for that real one-file scripts-only commit, control 17854cba0f (a packages/ commit) returns 14, so the probe is live. NOT RUN LOCALLY and left to CI: the full Test Core matrix — this diff is one workflow file and the farm runs it anyway.",
      "open_questions": [],
      "out_of_scope_findings": [
        "filed as #10015: ci.yml's core filter misses ten more declared cross-package test inputs — content/ (7 declarations, docs=true so those PRs look covered), docs/audits ledger, skills/objectstack-formula/**, .github/workflows/scaffold-e2e.yml; full census + per-root price table, argues for deriving filter entries from CROSS_PACKAGE_TEST_INPUTS once PR #9826 lands rather than adding four roots by hand",
        "filed as #10016 (finding, Blocked-by PR #10014): sync-template-versions.mjs's GAP 1 header says a diff confined to it means 'Test Core is skipped in full and the vitest never runs' — the scheduling half stops being true when this PR lands; comment-only fix, kept out of this PR because it would pull check:template-version-sync into a one-workflow-file diff"
      ]
    }

    Generated by Claude Code

  6. claude commented on Aug 19, 2026

    @claude
    Contributor

    Measurement artifacts for #9829 — PR #10014 (draft)

    Companion to the report comment above. Everything here was measured on branch
    claude/issue-9829-scripts-paths-filter @ e9ce81d9d2, and every filter evaluation was run under
    both picomatch 2.3.1 (what dorny/paths-filter@v4 actually bundles — its committed
    package.json says "picomatch": "^2.3.1", its package-lock.json resolves 2.3.1) and picomatch
    4.0.5 (what this tree has, and what #9880 measured against). The two agreed on every row of every
    table below
    , including apps/!(docs)/**. The version note in this card's PM comment is therefore
    discharged — and it mattered: the version to validate against is not the one in the tree.

    H1 — before/after pair

    BEFORE (origin/main 6de17ea93a)                    AFTER (PR #10014)
    changed : scripts/sync-template-versions.mjs       changed : scripts/sync-template-versions.mjs
    outputs : docs=false core=FALSE console=false      outputs : docs=false core=FALSE console=false scripts=TRUE
    test if : !cancelled() && core != 'false'          test if : !cancelled() && (core != 'false' || scripts != 'false')
    subst.  : true && "false" !== 'false'              subst.  : true && ("false" !== 'false' || "true" !== 'false')
    ⇒         Test Core SKIPPED                        ⇒         Test Core RUNS
    

    Monotone — the full truth table over the three values GitHub can deliver:

    core scripts before after
    true any RUNS RUNS
    false true SKIPPED RUNS
    false '' (filter job died) SKIPPED RUNS
    false false SKIPPED SKIPPED
    '' any RUNS RUNS

    Exactly one input class flips, towards running; #4928's "skip only when the filter EXPLICITLY said
    false" holds on both operands.

    H2 — the full per-top-level-entry table, with verdicts

    One real tracked file changed per top-level entry.

    changed file docs core console verdict
    scripts/… false false false WRONG — this card, fixed by PR #10014
    packages/… false true false correct
    examples/… false true false correct
    package.json false true false correct
    tsconfig.json false true false correct
    pnpm-lock.yaml true true false correct
    apps/docs/… true false false correct — Build Docs owns it
    content/… true false false WRONG — 7 declared test inputs live here (#10015)
    docs/… false false false WRONG for one path — docs/audits/2026-07-unknown-key-strictness-ledger.md (#10015)
    skills/… false false false WRONG for one path — skills/objectstack-formula/** (#10015)
    .github/… (not ci.yml) false false false WRONG for one path — .github/workflows/scaffold-e2e.yml (#10015)
    turbo.json false false false arguably wrong — defines every task's inputs/graph; named and priced, not added
    pnpm-workspace.yaml false false false covered in practice — such an edit moves pnpm-lock.yaml, which is in core
    .objectui-sha false false true correct — Console Pin Gate owns it
    .changeset/… false false false correct — changeset gates live in unfiltered lint.yml
    .claude/…, .githooks/…, .vscode/…, docker/…, root *.md, eslint.config.mjs, tsup.config.ts, lychee.toml, osv-scanner.toml, paseo.json, objectstack.code-workspace, .npmrc, .nvmrc, .gitignore, .gitattributes, .lycheeignore, .mcp.json false false false correct — every gate over these runs in unfiltered lint.yml

    The systematic version — instantiate each of turbo.json's 58 $TURBO_ROOT$/… declared test inputs
    to a real tracked file and run it through the core filter:

    unschedulable declared test inputs (core=FALSE), by top-level root:
      content    7      scripts    6      docs   1      skills   1      .github   1
    16 of 58 declared $TURBO_ROOT$ test inputs never schedule Test Core.
    

    One hole or a pattern: a pattern. The declaration layer knows five roots, the scheduling layer
    knows two. The scripts/ six are this card; the other ten are #10015.

    Note the card's own count moved: the three scripts/-rooted declarations it names are now six
    (scripts/** for spec, check-nul-bytes.mjs for cli, check-durability-degradation-log-level.mjs
    for metadata-protocol, and sync-template-versions.mjs + gen-sdui-manifest.sh +
    publish-smoke.sh for create-objectstack).

    turbo.json — no edit needed, with the measurement

    Its $TURBO_ROOT$/scripts/… entries are Layer B (task hash), and Layer B was never broken; only
    Layer A (package selection) was, and only because the job it lives in never started.

    • Layer B, live on this tree (turbo 2.10.10): create-objectstack#test hashes 816a14410f216396
      clean, 14e2a95d6e6747de with a one-line no-op appended to scripts/sync-template-versions.mjs,
      and 816a14410f216396 again after restoring the file — the restore leg proves the probe, not just
      the mutation.
    • Layer A, by running the real gate against a zero-package payload: + @objectstack/spec for any
      scripts/ path, + create-objectstack for sync-template-versions.mjs, + @objectstack/cli for
      check-nul-bytes.mjs.
    • The zero-package payload is measured, not assumed: TURBO_SCM_BASE=52db5df65^ TURBO_SCM_HEAD=52db5df65 turbo ls --affected returns {"count":0,"items":[]} for that real
      one-file scripts/-only commit; control 17854cba0f (a packages/ commit) returns 14.
    • pnpm check:cross-package-test-inputs is green: "12 package(s) read outside themselves, all
      declared, and turbo.json hashes every declared glob."

    H4 — cost, and why the narrower filter does not exist

    Over the last 100 first-parent commits on origin/main:

    candidate commits touching it of those, core=false today (⇒ newly run Test Core)
    scripts/** (shipped) 44 30
    content/** 16 6
    .github/workflows/** (other than ci.yml) 11 6
    docs/** 5 2
    skills/** 2 2
    turbo.json 3 1
    pnpm-workspace.yaml 1 0

    30% is a large fraction, and the per-run cost is larger than the triage note assumed: turbo ls --affected does return zero packages, but the union then adds @objectstack/spec for every
    scripts/ path (spec declares scripts/**), and spec has 414 test files — a cold
    turbo run test --filter=@objectstack/spec on this shared 4-vCPU container had not finished after 7
    minutes. So it is one shard doing real work plus checkout/install on three runners, not a near-empty
    job.

    The narrower filter H4 asks for does not exist at this layer: the union of the six
    scripts/-rooted declarations is scripts/**, so a declaration-derived filter is byte-identical
    to what shipped, and any hand-narrowed list is a second recognizer disagreeing with the declaration
    it serves (#9747 family, and the "list you must remember to update" shape of #7802). The route to a
    genuinely narrower filter runs through narrowing spec's own scripts/** declaration in
    scripts/check-cross-package-test-inputs.mjs — the file open PR #9826 is rewriting. Recommendation:
    ship scripts/** as written.

    Ruling 1 — parsed structural proof

    Both versions parsed with the yaml package and compared job by job (not an eyeball of the diff):

    job ids identical            : true  (10 jobs)
    triggers (on:) identical     : true
    concurrency identical        : true
    every job `name:` unchanged  : true   ← the check-run names branch protection matches on
    every step count unchanged   : true
    every `run:` line unchanged  : true
    every `uses:` unchanged      : true
    docs / core / console pattern lists unchanged : true / true / true
    every job `if:` unchanged except `test`       : true
    

    The only two fields that differ anywhere in the file: filter.outputs gains scripts, and
    test.if gains the ORed operand. No job renamed, no step added or removed — no required context
    detaches.


    Generated by Claude Code


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions