Repository navigation
ci(refresh lane): each test-timing refresh diffs the window's per-package weights against the previous dataset — a package up 25 % or more, or a file over the 10 s budget, is a filed card for its lane, never a red build (#22949 ruling E, items 2–3) #23059
Description
Protects: the only regression detector the test suite has once the wall ceiling stops grading (#22949 ruling E): the refresh dataset's own history, read in the same density window to window.
Filed by the director seat (summon #37, session_01LqNgdPZB2Jnr5HpGbaqLeB) on the maintainer's word 「22949 同意」 (2026-10-12) to this seat's letter E on #22949 (items 2 and 3 of the ruling, its newest Ruling: comment). Filing gate ③: maintainer-directed. ⛔ Not a claim. Lands after E1 (the report-only card) or in parallel on disjoint files (scripts/measure-test-shard-timings.mjs, the refresh workflow, the refresh PR body template); the dev states which.
What to build
- The diff:
scripts/measure-test-shard-timings.mjs(or a sibling the refresh workflow calls) compares the new window's per-package weight with the previous dataset's (scripts/test-shard-timings.jsononmainbefore the refresh): packages up by 25 % or more (the ceiling's own factor) are listed in the refresh PR's body with both numbers, the window's density reading (how many packages executed per run, replays excluded) and the slowest file of each; packages down are listed as a one-line summary. - Density guard: the diff compares like with like: the window's runs are tagged by executed-package count, and a package whose rise is explained by density alone (its share of the run's executed total unchanged within the factor) is not a finding; the first diff after this ruling baselines on the first window of comparable density and says so.
- The per-file budget: the refresh PR body lists every test file over 10 s of wall in the window (objectstack#22998's rule), with its package and its slowest run; the dataset already carries the
Slowest file:reading, so this is a read, not a new measurement. - Card filing: the refresh lane's seat files one card per package named by the diff (the owning lane's
domain:*,Protects:line naming the package and the numbers), never a red build; the filing is the seat's act after the refresh PR lands, written into the refresh lane's instructions (its runbook or the workflow's header), not automated by the bot. - ⛔ Not touched: the ceiling table's formula, the floor,
RULED_CEILING_RAISES, the two timing steps (E1).
Acceptance
- The first refresh after landing carries the three lists in its PR body (rises, files over budget, density per run); its self-test covers a density-only rise (not a finding) and a code rise (a finding); the refresh lane's instructions say who files what.
Related: #22949 (ruling E) · E1 · #22998 (test budgets) · #16468 · #22923 (the first dataset with ceilings) · scripts/measure-test-shard-timings.mjs · scripts/test-shard-timings.json.
Activity
objectstack-fleet commentedon Oct 12, 2026
Claim: PM loop round 1 · the director seat dispatching on the maintainer's words, verbatim 「22949 同意」 (ruling E, 6125775640) and the standing order 「三仓的 ci 优化卡你负责派发」 (this seat's session, 2026-10-12T14:42Z)
Session:
session_01LqNgdPZB2Jnr5HpGbaqLeBAccount:
os-zhuangBranch:
claude/issue-23059-refresh-diff-and-file-budgetWorktree:
objectstack-issue-23059Domain:
domain:devxSeat:
domain:devx#9File surface:
scripts/measure-test-shard-timings.mjs(or a sibling it calls: the window diff, the density tag, the per-file list) and its self-test,scripts/ci/select-shard-timings-run.mjsonly if the window selection must carry the density tag,.github/workflows/shard-timings-refresh.yml(the refresh PR body and the lane's instruction lines only), the refresh lane's runbook or header where the card-filing duty is written; ⛔ not the ceiling formula, the floor,RULED_CEILING_RAISES, the two timing steps (E1 = #23058, in flight),ci.yml,lint.yml(stop on breach; explain in the report)Container & model:
M,mode:subagent,model: opus(the default judgment tier: a measurement design with a density guard). Tier H posture:.github/workflows/**(the round's posture): the PR stays a draft until aGOVERNED_APPROVERSmember approves it.Clause-②: no
Responsibility:
n/a — not a defect cardThread-read: 6125775640
Ruling-ref: #22949 ruling E 6125775640 (items 2 and 3; the maintainer's word verbatim 「22949 同意」), objectstack#22998 (test budgets: a file over 10 s is a card), #16468 (the ratchet the dataset serves), #22923 (the first dataset with ceilings)
Serial constraints cleared: #23058 (E1, in flight) edits
partition-test-shards.mjs,check-test-suite-ceilings.mjsand two step comments inci.yml, none of this claim's files; #23044 (claimed bydomain:devxseat 1) editslint.ymland a new size gate; no other in-flight claim declares the refresh lane's files. Mergeorigin/mainbefore opening the PR.objectstack-fleet commentedon Oct 12, 2026
os-dev-report
{
"issue": 23059,
"status": "done",
"branch": "claude/issue-23059-refresh-diff-and-file-budget",
"pr": "#23066",
"session": "session_01LqNgdPZB2Jnr5HpGbaqLeB — the dispatching seat's session, read from the container by the relay (subagent = parent's)",
"premise_still_valid": true,
"summary": "Ruling E items 2-3 implemented, draft PR #23066 (assignee os-zhuang, labels ci/cd, size/xl, skip-changeset; never readied, queue never armed). The generator now records a
densityfield per refresh (each run's count of EXECUTED packages, replays excluded, and each package's median share of its runs' executed seconds) and has a--diffmode that renders the refresh PR's regression section: rises of 1.25x or more against the previous dataset with both numbers, share before/after, slowest file and a verdict (finding / density only / provisional / no prior share / baseline), downs in one line, every test file over 10 s with its package and slowest run, the packages lacking a per-file reading and why, and a "What the seat landing this PR does" section with ready Protects: lines. The workflow header now states who files what (the landing seat, one card per finding for the owning lane, never the bot, never a red build); a new body-only step downloads the window's per-file captures; the compose step puts the section at the top of the body. Sub-premise FALSE: the dataset carries no per-file reading (nor per-run density);Slowest file:comes from the per-shardtest-core-timings-N-of-6captures of the same runs (same 1-day retention, measured on run 38195222269), so the per-file list reads those captures. Claim check: the newest Claim: (6126539555) names this branch; no second claim posted; card assignee not written.","tests": "All on head 72443a3.
node scripts/measure-test-shard-timings.mjs --self-test-> "self-test OK" exit 0; new battery "refresh diff and per-file budget (#22949 E)" floor 22 = registered (floor 23 measured red), roster floor 4->5; first two cases are the acceptance fixtures: density-only rise (4->5 executed/run, every suite 1.3x, a up 1.3x, share 0.81x -> density only) and code rise (same density, a 2.5x, share 2.17x -> finding).node scripts/ci/select-shard-timings-run.mjs --self-test-> "self-test OK" exit 0; first battery floor 35->37 = registered; new battery "shard-timings-refresh per-file captures (#22949 E)" floor 5 (roster 5->6) drives the new download step's own run: block under bash -e with stub curl/unzip (one failed download, one run without captures) then the REAL --diff over what landed. Ablations via scripts/ablation-replace.mjs, each restored with git diff HEAD empty: (1) density guard removed -> generator self-test exit 1 "diff: a rise explained by density alone was read as finding (findings 4)"; (2) download step made to exit 7 on a failed capture -> selector self-test exit 1 "captures step: a missing capture failed the step (exit 7)" — the first attempt at (2) was a NO-OP refused by the tool (replacement contained the anchor, anchor count 1->1), rerun with a disjoint anchor. actionlint 1.7.7 on the workflow exit 0; control leg (a step if: naming an undefined step id) reported. No shellcheck on the box: step scripts NOT MEASURED by shellcheck. YAML parse (yaml pkg): 16 steps, triggers byte-unchanged. No package build/typecheck owed: the diff touches no workspace package (root scripts/ and .github/ only).","gates": "
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack(no paths, merge base 0cbfb7c): 51 commands; all run on 72443a3, all exit 0 (50 via a sequential runner with per-command logs,pnpm check:pm-dispatch-gatesdetached via nohup with its exit captured: 0, fast tier 1833 cases).--ranwith every exit code recorded: "51 derived famil(ies) accounted for — 51 run, 0 NOT-MEASURED (a DERIVED zero)".check:declared-population-livewas RED on 15c0880 (the capture schema constant was a quoted path-shaped module-scope literal reaching no tracked file) and green after the fix (see deviations);check:nul-bytesexit 0. Re-derived list on the final head identical to the first. CI on the PR: in_progress at report time (the PR's own "Regenerate the shard-timings dataset" dry run was running).","files_changed": [
"scripts/measure-test-shard-timings.mjs",
"scripts/ci/select-shard-timings-run.mjs",
".github/workflows/shard-timings-refresh.yml"
],
"density_definition": "Run tag = the run's count of EXECUTED packages (cache MISS, exit 0, complete slice set within the run; replays excluded; a run that executed nothing has no tag). Window density = the MEAN tag over the window's runs (mean, not median: hourly runs are bimodal). "Comparable" = the two windows' means within 1.25x of each other (DENSITY_COMPARABLE_FACTOR = CEILING_HEADROOM; 20 vs 25 comparable, 20 vs 25.1 not, both pinned in the self-test). Density guard (ruled) = a rise whose share of its runs' executed total moved by less than 1.25x is "density only", not a finding. Findings are judged only between comparable windows whose previous dataset carries a density reading; otherwise the diff is a BASELINE and says why. First diff after landing: previous dataset has no
density-> BASELINE over the runs that refresh accumulates (each tagged in the body), taken because they are the first runs a density can be read from.","replay": "measure --diff, prior = #22368 dataset (0401847, 2026-10-08, 21 runs), refreshed = #22923 dataset (1bd612b, 2026-10-12, 12 runs): "12 rise(s) of 1.25x or more, 0 finding(s) (BASELINE: none judged); 33 down; files over budget NOT MEASURED". Rises (all baseline): trigger-api 2.18->23.85 (10.94x), example-todo 1.53x, example-showcase 1.51x, service-realtime 1.42x, service-cluster-redis 1.35x, plugin-auth 359.89->465.63 (1.29x), organizations 1.29x, metadata-core 1.29x, plugin-dev 1.29x, plugin-pinyin-search 1.28x, connector-slack 1.27x, service-settings 1.25x. Downs: 33, 12 to 80 % or less (cloud-connection 153.64->81.68, service-analytics 401.06->220.50, formula, client, connector-rest, sdui-parser, service-cluster, driver-sqlite-wasm, plugin-hono-server, example-crm, +2). Neither dataset carries density, so the replay is a BASELINE by construction; the first live diff differs only in its reason ("the previous dataset carries no density reading") and in having a measured files list.",
"not_measured": [
"The live refresh: the lane writes to main on its own Monday schedule; the first scheduled refresh after merge is the measurement.",
"Closest live reading: the PR's own pull_request dry run of shard-timings-refresh.yml renders the would-be body (BASELINE, real density tags, real files over budget) into its step summary — web UI only (no REST endpoint for a job summary); the --diff verdict line and the captures read are in its job log. In progress at report time; the PM reads its conclusion.",
"Density and per-file readings for the two replayed windows: #22368's artifacts are past 1-day retention; #22923's captures are retained but this container cannot download an artifact (GET /actions/artifacts/11723746884/zip -> blob host 403).",
"shellcheck over the new step scripts (not installed)."
],
"deviations": [
"Selector touched (claim allowed it only for the density tag): runIsEligible/--candidates now carry
timing_artifact_ids, the per-file read's input; not an eligibility input.","New workflow step "Download the window's per-file timing captures for the pull request body" (claim scoped the workflow to the PR body and instruction lines): body-only, cannot fail the job; needed because the dataset carries no per-file reading.",
"Director relay applied: the compose step's over-ceiling sentence no longer says "the brake fired on main"; it reads "a reading, not a red -- under #22949 ruling E the ceiling reports on every event, and a regression is judged by the diff at the top of this body". The generator's matching code comment reworded the same way. Commit 72443a3 (the compose step was already edited by 52e2ba4, so this landed as a follow-up commit, not the same one).",
"measure-test-shard-timings.mjs now carries a
dispatch-gates: no-path-populationmarker and spells the capture schema as a join, the remedy check:declared-population-live prescribes; the family is still placed by its own file (re-derived).","Card-filing interpretation written into the lane's instructions: files over budget = one card per owning package naming its files (not one per file), and an open card gets the new numbers as a comment instead of a second card. Named in the PR's Acceptance notes for the seat to confirm or strike."
],
"mcp_calls": "0 — no MCP GitHub tool used.",
"api_writes": "3 relay dispatches from this container, each POST /repos/objectstack-ai/objectstack/dispatches (HTTP 204), executed as objectstack-fleet[bot]: (1) pr_create -> POST /repos/objectstack-ai/objectstack/pulls (draft forced) + POST /repos//issues/23066/assignees (os-zhuang), body read back 15794 bytes identical; (2) label-write labels_add -> POST /repos//issues/23066/labels (skip-changeset), read back: ci/cd, size/xl, skip-changeset; (3) this os-dev-report comment -> POST /repos//issues/23059/comments. Plus git push (not REST). Reads: single-card REST reads of #23059, its comments, ruling comment 6125775640, #22998, PR #23066, and Actions run/artifact/annotation metadata.",
"open_questions": [],
"out_of_scope_findings": [
"carrier: 承接者:无 · noted, not filed — the refresh workflow header's older paragraph says
Test Core"grades every executed package" against the ceilings; under ruling E it reports. Stale copy, not a defect class; in the PR's Acceptance notes."]
}
objectstack-fleet commentedon Oct 12, 2026
Review: ACCEPT · PR #23066 (draft, head
72443a3c) · director seat dispatching on the maintainer's order, 2026-10-12T15:38ZChecked on GitHub and on the head's tree, not on the report's say-so:
main(merge-base0cbfb7c3), body line 1Fixes #23059(the only closing keyword),Clause-②: no, assigneeos-zhuang, labelsci/cd,size/xl,skip-changeset; 3 files, +1,360/−14, six commits; the newestClaim:(6126539555) names the branch and the queue guardsThe card this PR closes must claim this branchandNo other open PR may claim the same issueare green.measure-test-shard-timings.mjs,select-shard-timings-run.mjsandshard-timings-refresh.ymlonly;check-test-suite-ceilings.mjs(formula,CEILING_FLOOR_SECONDS,RULED_CEILING_RAISES),partition-test-shards.mjs,ci.ymlandlint.ymlare untouched. The refresh workflow'son:block (schedule30 5 * * 1,workflow_dispatch,pull_requeston its three paths) and its write step are byte-unchanged; the one new step (Download the window's per-file timing captures for the pull request body,if: steps.compare.outputs.changed == 'true') guards every command that can fail with a::warning::andcontinue, so it cannot red the job; the compose step runs--diffagainstgit show HEAD:scripts/test-shard-timings.jsonand, on a refusal, puts the refusal text in the body's place and still opens the PR.RISE_FACTOR=DENSITY_COMPARABLE_FACTOR=CEILING_HEADROOM(1.25),FILE_BUDGET_SECONDS= 10 (lines 175, 219–221); the five verdictsfinding/density only/provisional/no prior share/baseline(lines 1037–1041); comparable density is a number (two windows' mean executed-package count within 1.25×; the self-test pins 20 vs 25 comparable and 20 vs 25.1 not, and 4 vs 5 per run comparable at line 2607); the density guard: a rise whose share moved by less than 1.25× isdensity only; the first diff after landing is a BASELINE and the body says why. The header sectionREGRESSION DETECTION LIVES HERE, AND WHO FILES WHAT(workflow lines 158–200) says the bot files nothing, nothing is ever a red build, and the seat landing the refresh PR files one card perfindingwith aProtects:line; the body's ownWhat the seat landing this PR doessection repeats it with the run's list.72443a3cover the chore(ci): refresh the Test Core shard-timings dataset #22368 dataset (04018475) and the chore(ci): refresh the Test Core shard-timings dataset #22923 dataset (1bd612bd): exit 0,12 rise(s) of 1.25x or more, 0 finding(s) (BASELINE: none judged); 33 down; files over budget NOT MEASURED; the rows match the PR body (trigger-api2.18 → 23.85 s, 10.94×;plugin-auth359.89 → 465.63 s;service-settings73.71 → 92.33 s); the body's four sections render as described.densitykey breaks no reader:partition-test-shards.mjsloadTimingsreadspackagesandsecondsPerTestFileFallbackand ignores the rest;check-test-suite-ceilings.mjsreadsceilings;report-test-timings.mjsreads the pinned weights; none validates the key set.timing_artifact_idson the selector's candidates sits outsiderunIsEligible'sreasons(the tworeasons.pushsites are the jobs and the run-summary artifacts).72443a3c, read at this act: green:Lint & Repo Gates(withLint · PM dispatch-gates,scoped · tooling self-tests,repo & tooling gates,docs & code guards,CI, release & engine gates,slot lookup & corpus,query-options erasure,ESLint),TypeScript Type Check(workspace, source gates, consumer gates, debt ledger),Test Core2/6–6/6,Governed Surface Queue Guard,Check Changeset,Check PR Size, the three claim and single-writer guards, andRegenerate the shard-timings dataset(the lane's ownpull_requestdry run: the real selection, regeneration, capture download and--diffagainstmain's dataset, its would-be body in the step summary; the seat cannot read a job summary over REST, so the body's content from that run is the maintainer's reading in the web UI); still running:Test Core (1/6)and its aggregator; skipped by design:Build Core,Build Docs(no package touched). Nothing red. The dev's local union: 51 derived, 51 run, all exit 0,--ran0 unrun; both self-tests OK with their floors raised to the registered counts; two ablations red in the predicted direction.check-governed-merges.mjs --pr 23066finds no register row (1,374 lines ≤ 3000); the round's Tier H posture for.github/workflows/**is the maintainer's word, so the PR stays a draft until aGOVERNED_APPROVERSmember approves, then the seat readies and arms the queue. Contract review: none of the three faces touched → the seat's own reading, no second agent.Slowest file:comes from the per-shardtest-core-timings-N-of-6captures of the same runs, retained as long as the run summaries (measured on run 38195222269). The per-file list is still a read of an existing measurement, not a new one; the two deviations it forced (the selector carries the capture ids; the download step) are proportionate and named.dispatch-gates: no-path-populationmarker and the joined capture-schema string (the remedycheck:declared-population-liveprescribes); the "brake fired on main" sentence rewritten as a reading (the seat's relay from the ci(timing gates): the two timing steps report-only on every event,scheduleincluded — the per-package wall ceiling and the shard drift stay readings, never a red (#22949 ruling E, item 1) #23058 dev). Writes: 3 relay strokes (pr_create, label-write, the report 6127459459) and the pushes;mcp_calls0.Test Core"grades every executed package" (stale under ruling E; cosmetic; carrier none).Landing: on
Test Coregreen and the maintainer's Approve, the seat readies the PR and arms the queue. NOT MEASURED until then: a live refresh. The lane writes tomainon its Monday 05:30 UTC schedule; the first refresh after landing is a BASELINE (the previous dataset carries nodensity) and its body carries the three lists with the first real density tags and files over budget; the seat reads it, files the per-package cards for files over 10 s, and judges nothing until the next comparable window. This card stayspm:dispatcheduntil the landing record.objectstack-fleet commentedon Oct 12, 2026
LANDED — PR #23066 is MERGED (2026-10-12T21:57Z, approved by the maintainer, review 5496222580 on head
72443a3c, readied, auto-merge armed and enqueued by hand; merge commitd0f1c5bf) and read in place onorigin/maind0f1c5bf. Director seat, summon #37,session_01LqNgdPZB2Jnr5HpGbaqLeB, 2026-10-12T21:58Z.origin/maindensity(densityOf(runReadings)) and exportsdiffRefreshRISE_FACTOR=DENSITY_COMPARABLE_FACTOR=CEILING_HEADROOM,FILE_BUDGET_SECONDS= 10timing_artifact_idsREGRESSION DETECTION LIVES HERE, AND WHO FILES WHAT, the capture-download step,--diffin the compose step, the cron30 5 * * 1unchangedcheck-test-suite-ceilings.mjs,partition-test-shards.mjs,ci.ymluntouched by this merge (diff against56986e9d)mainstill carries nodensity(the first refresh after this is a BASELINE)densityFixesline (closed_by_pull_requestslists PR ci(refresh lane): each refresh diffs its window against the previous dataset, density-guarded, and lists test files over 10 s; the lane's seat files the cards #23066 alone);pm:dispatchedstripped and the assignee released in this act. Items 2–4 of ruling E on ci: merge-queueTest Corereds on the new suite-duration ceiling —@objectstack/cloud-connectionmeasured 177.5 s and 200.3 s vs its 163.89 s ceiling, in two groups of PRs that do not touch it (live since1bd612bd6) #22949 are onmain; with ci(timing gates): the two timing steps report-only on every event,scheduleincluded — the per-package wall ceiling and the shard drift stay readings, never a red (#22949 ruling E, item 1) #23058's landing (56986e9d) the whole ruling is executed.density), the window's density tags, the rises as readings, and the test files over 10 s from the window's captures. The seat reads it, files one card per owning package for the files over budget (the interpretation confirmed in ACCEPT 6127535812; the maintainer may strike it) and judges no rise until the next comparable window.Fixes #23059alone.