Skip to content

Commit ea7ff39

Browse files
chore(ci): refresh the Test Core shard-timings dataset (#20388)
Refreshes `scripts/test-shard-timings.json`, the balancing input for the Test Core shard split. Opened automatically by `.github/workflows/shard-timings-refresh.yml`. Every byte came out of `scripts/measure-test-shard-timings.mjs`; nothing here was hand-edited, and no bound, timeout or matrix entry was touched. ## Source Measured across 1 accumulated run(s) of the HOURLY `schedule` run of CI on `main` — the full-battery run (#16467). A `push` run on `main` is affected-only and is not a measurement of the workspace, so no push run feeds this file. No single green run measures the whole workspace either — turbo's cache is namespaced per shard and only main pushes write it, so a package whose inputs have not changed is a HIT and the generator refuses hits rather than recording a replay as a duration. Runs are therefore accumulated, each fenced by its own `--run` group, until every package the committed dataset holds is measured again; a package seen in several of them gets the median of those observations. - https://github.com/objectstack-ai/objectstack/actions/runs/36380128221 - Newest run in the set: `36380128221`, commit `eee0974236c87a7232f9805a19b6e53e3e36da59` — the date this refresh carries. - Every run above had all six `Test Core (N/6)` jobs conclude `success` with its six run-summary artifacts still retained; runs that were cancelled, failed or had lost their artifacts were rejected by name in the log before any of these were used. All 72 package weights were measured in these runs; nothing was carried. ⚠️ This PR references #16173 and #16222 but does NOT carry a closing keyword for them, because a weekly lane cannot know which cards a given run ought to retire. If this is the first refresh to land, retire those two by hand as part of merging it. ## Measured per-shard suite time on the newest run in the set ``` 1: 1023s | 2: 757s | 3: 957s | 4: 728s | 5: 728s | 6: 619s ``` ## Predicted bins, before and after ``` BEFORE partition-test-shards: self-test OK (70 measured packages -> 71 shard items, 6 shards, max/mean 1.00x <= 1.3x, floor 404s, bins 666/666/666/667/666/666s) AFTER file:///home/runner/work/objectstack/objectstack/scripts/partition-test-shards.mjs:967 throw new Error( ^ Error: slice derivation: UNSLICED, @objectstack/cli at 1231.52s is 1391s against a 1321s mean and now fits under 1.3x on its own, so the two cases above no longer prove the slicing is what satisfies the bound. Re-derive the slice count (or retire it) instead of leaving a pin that cannot fail. at file:///home/runner/work/objectstack/objectstack/scripts/partition-test-shards.mjs:967:15 at check (file:///home/runner/work/objectstack/objectstack/scripts/partition-test-shards.mjs:694:5) at selfTest (file:///home/runner/work/objectstack/objectstack/scripts/partition-test-shards.mjs:965:5) at main (file:///home/runner/work/objectstack/objectstack/scripts/partition-test-shards.mjs:1513:9) at file:///home/runner/work/objectstack/objectstack/scripts/partition-test-shards.mjs:1579:3 at ModuleJob.run (node:internal/modules/esm/module_job:343:25) at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:681:26) at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:117:5) Node.js v22.23.2 ``` ## The partitioner's own pins RED on this refresh — read this before merging This is the designed behaviour, not a defect in the refresh: the acceptance bound is a ratio, and a package that has grown past what any six-way split can bin makes the pins fail with the arithmetic in the message. The remedy the partitioner names is to raise the file-level slice count for that package — ⛔ never to raise the bound, and ⛔ never to hand-edit this dataset. This workflow deliberately does neither: it reports and stops, because both are decisions. ## No checks will start on this PR by themselves It was opened with the Actions `GITHUB_TOKEN`, and GitHub's recursion guard means a PR opened that way triggers no workflow runs. Push any commit to the branch, or close and reopen the PR, to start CI. Refs #16464, #16173, #16222. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: os-sales <sales@objectstack.ai>
1 parent 8cbba54 commit ea7ff39

1 file changed

Lines changed: 91 additions & 82 deletions

File tree

‎scripts/test-shard-timings.json‎

Lines changed: 91 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -1,90 +1,99 @@
11
{
2-
"note": "GENERATED by scripts/measure-test-shard-timings.mjs -- do not hand-edit. Per-package `turbo run test` durations in seconds, the balancing input for the Test Core shard split (scripts/partition-test-shards.mjs). See `provenance.refresh` to regenerate.",
2+
"note": "GENERATED by scripts/measure-test-shard-timings.mjs -- do not hand-edit. Per-package `turbo run test` durations in seconds, the balancing input for the Test Core shard split (scripts/partition-test-shards.mjs). See `provenance.refresh` to regenerate. Every weight is a real measurement: the ones in `carriedOver` were measured by an earlier refresh and re-confirmed unchanged by a turbo cache HIT in this one.",
33
"provenance": {
4-
"measuredAt": "2026-08-24",
4+
"measuredAt": "2026-09-28",
55
"summaries": [
6-
"3INZWsDPxzZHTiHyMLtMaW3QzzS.json",
7-
"3INauIhwAvRiujuKKwByt4oUHia.json",
8-
"3INblkezuMcTpRYH7dacKvoCrKN.json",
9-
"3INcmCJVPcuFrxIblRK7pEmTaM9.json",
10-
"3INddg5OSvLXaJIzH4NoHlu5lUk.json",
11-
"3INdx8H55OYIsI3ezcpzRBx3GDO.json"
6+
"3JwWrbZiCOcylFiiB35ZkJOifXv.json",
7+
"3JwWH46mvzYLLstazVgnThFBQF7.json",
8+
"3JwWDxW3nyzFgBFRPzsbWsJLCTc.json",
9+
"3JwWyHOCxXLi8SczV1jbpNWxDO8.json",
10+
"3JwWhAnQd01tDSRm9u9uKPsw0Ql.json",
11+
"3JwW4l346RhKYwvosEmVvhQNPVz.json",
12+
"3JwWoVaaflhUD2yWSIIt4evAFhy.json",
13+
"3JwWCoR24UAY40DcD8bYlUWcdn0.json"
1214
],
13-
"mergeRule": "median across summaries",
14-
"refresh": "node scripts/measure-test-shard-timings.mjs <run-summary.json>... --out scripts/test-shard-timings.json (summaries: the `test-core-run-summary-<n>-of-6` artifacts of any green merge_group run, or a local `pnpm exec turbo run test --concurrency=4 --summarize`)"
15+
"runs": [
16+
"36380128221"
17+
],
18+
"mergeRule": "median across runs; a file-sharded package is summed from its slices WITHIN one run first, and a run that cannot assemble every slice contributes no sample for it; a package not measured in this pass keeps its previous weight ONLY when a turbo cache HIT witnesses that its inputs are unchanged, and every such package is named in `carriedOver`",
19+
"refresh": "node scripts/measure-test-shard-timings.mjs [--run <id>] <run-summary.json>... --merge-into scripts/test-shard-timings.json --out scripts/test-shard-timings.json (summaries: the `test-core-run-summary-<n>-of-6` artifacts of any green run, or a local `pnpm exec turbo run test --concurrency=4 --summarize`. ⚠ ONE RUN COVERS ONLY 2-52 of ~71 packages depending on cache warmth, so feed SEVERAL runs -- a `--run <id>` before each run's summaries is REQUIRED, so a sliced package is assembled per run and then medianed like every other package -- and `--merge-into` to carry the cache-hit remainder. `.github/workflows/shard-timings-refresh.yml` does all of this weekly.)"
1520
},
16-
"secondsPerTestFileFallback": 1.522,
21+
"secondsPerTestFileFallback": 2.132,
1722
"packages": {
18-
"@objectstack/cli": 458.15,
19-
"@objectstack/client": 32.29,
20-
"@objectstack/client-react": 8.12,
21-
"@objectstack/cloud-connection": 33.64,
22-
"@objectstack/connector-mcp": 9.95,
23-
"@objectstack/connector-openapi": 10.31,
24-
"@objectstack/connector-rest": 8.29,
25-
"@objectstack/connector-slack": 9.52,
26-
"@objectstack/core": 15.1,
27-
"@objectstack/downstream-contract": 9.88,
28-
"@objectstack/driver-memory": 51.03,
29-
"@objectstack/driver-mongodb": 40.03,
30-
"@objectstack/driver-sql": 203.97,
31-
"@objectstack/driver-sqlite-wasm": 57.96,
32-
"@objectstack/driver-turso": 76.52,
33-
"@objectstack/embedder-openai": 3.51,
34-
"@objectstack/example-crm": 27.7,
35-
"@objectstack/example-embed-objectql": 19.6,
36-
"@objectstack/example-showcase": 22.87,
37-
"@objectstack/example-todo": 30.46,
38-
"@objectstack/formula": 12.39,
39-
"@objectstack/hono": 5.07,
40-
"@objectstack/http-conformance": 40.88,
41-
"@objectstack/knowledge-memory": 1.39,
42-
"@objectstack/knowledge-ragflow": 1.44,
43-
"@objectstack/lint": 86.66,
44-
"@objectstack/mcp": 34.44,
45-
"@objectstack/metadata": 43.03,
46-
"@objectstack/metadata-core": 9.38,
47-
"@objectstack/metadata-fs": 30.54,
48-
"@objectstack/metadata-protocol": 278.86,
49-
"@objectstack/objectql": 222.2,
50-
"@objectstack/observability": 2.17,
51-
"@objectstack/platform-objects": 18.78,
52-
"@objectstack/plugin-approvals": 31.07,
53-
"@objectstack/plugin-audit": 36.37,
54-
"@objectstack/plugin-auth": 221.71,
55-
"@objectstack/plugin-dev": 14.67,
56-
"@objectstack/plugin-email": 39.1,
57-
"@objectstack/plugin-hono-server": 25.18,
58-
"@objectstack/plugin-pinyin-search": 14.4,
59-
"@objectstack/plugin-security": 114.92,
60-
"@objectstack/plugin-sharing": 62.69,
61-
"@objectstack/plugin-webhooks": 25.97,
62-
"@objectstack/rest": 236.68,
63-
"@objectstack/runtime": 264.23,
64-
"@objectstack/sdui-parser": 1.62,
65-
"@objectstack/service-analytics": 103.62,
66-
"@objectstack/service-automation": 128.89,
67-
"@objectstack/service-cache": 1.86,
68-
"@objectstack/service-cluster": 3.72,
69-
"@objectstack/service-cluster-redis": 3.61,
70-
"@objectstack/service-datasource": 32.19,
71-
"@objectstack/service-i18n": 10.05,
72-
"@objectstack/service-job": 8.44,
73-
"@objectstack/service-knowledge": 15.21,
74-
"@objectstack/service-messaging": 28.44,
75-
"@objectstack/service-package": 7.54,
76-
"@objectstack/service-queue": 16.03,
77-
"@objectstack/service-realtime": 3.66,
78-
"@objectstack/service-settings": 41.61,
79-
"@objectstack/service-sms": 11.02,
80-
"@objectstack/service-storage": 29.28,
81-
"@objectstack/spec": 403.65,
82-
"@objectstack/trigger-api": 1.72,
83-
"@objectstack/trigger-record-change": 23.53,
84-
"@objectstack/trigger-schedule": 13.49,
85-
"@objectstack/types": 5.69,
86-
"@objectstack/verify": 83.98,
87-
"create-objectstack": 15.5
23+
"@objectstack/cli": 733.33,
24+
"@objectstack/client": 115.44,
25+
"@objectstack/client-react": 7.36,
26+
"@objectstack/cloud-connection": 77.22,
27+
"@objectstack/connector-mcp": 24.51,
28+
"@objectstack/connector-openapi": 28.33,
29+
"@objectstack/connector-rest": 26.56,
30+
"@objectstack/connector-slack": 26.06,
31+
"@objectstack/core": 57.09,
32+
"@objectstack/downstream-contract": 15.41,
33+
"@objectstack/driver-memory": 104.62,
34+
"@objectstack/driver-mongodb": 66.07,
35+
"@objectstack/driver-sql": 283.54,
36+
"@objectstack/driver-sqlite-wasm": 69.46,
37+
"@objectstack/driver-turso": 137.33,
38+
"@objectstack/embedder-openai": 4.45,
39+
"@objectstack/example-crm": 30.95,
40+
"@objectstack/example-embed-objectql": 32.7,
41+
"@objectstack/example-showcase": 59.76,
42+
"@objectstack/example-todo": 35.3,
43+
"@objectstack/formula": 34.02,
44+
"@objectstack/hono": 31.26,
45+
"@objectstack/http-conformance": 67.43,
46+
"@objectstack/knowledge-memory": 3.54,
47+
"@objectstack/knowledge-ragflow": 2.97,
48+
"@objectstack/lint": 135.82,
49+
"@objectstack/mcp": 44.63,
50+
"@objectstack/metadata": 69.25,
51+
"@objectstack/metadata-core": 19.23,
52+
"@objectstack/metadata-fs": 36.24,
53+
"@objectstack/metadata-protocol": 341.53,
54+
"@objectstack/objectql": 359.61,
55+
"@objectstack/observability": 2.72,
56+
"@objectstack/organizations": 43.6,
57+
"@objectstack/platform-objects": 74.5,
58+
"@objectstack/plugin-approvals": 105.27,
59+
"@objectstack/plugin-audit": 50.91,
60+
"@objectstack/plugin-auth": 189.18,
61+
"@objectstack/plugin-dev": 41.7,
62+
"@objectstack/plugin-email": 49.79,
63+
"@objectstack/plugin-hono-server": 57.28,
64+
"@objectstack/plugin-pinyin-search": 38.77,
65+
"@objectstack/plugin-security": 278.19,
66+
"@objectstack/plugin-sharing": 98.82,
67+
"@objectstack/plugin-webhooks": 43.56,
68+
"@objectstack/rest": 240.77,
69+
"@objectstack/runtime": 434.86,
70+
"@objectstack/sdui-parser": 4.54,
71+
"@objectstack/service-analytics": 212.43,
72+
"@objectstack/service-automation": 235.74,
73+
"@objectstack/service-cache": 2.28,
74+
"@objectstack/service-cluster": 16.86,
75+
"@objectstack/service-cluster-redis": 15.5,
76+
"@objectstack/service-datasource": 61.84,
77+
"@objectstack/service-i18n": 19.78,
78+
"@objectstack/service-job": 17.67,
79+
"@objectstack/service-knowledge": 24.72,
80+
"@objectstack/service-messaging": 117.1,
81+
"@objectstack/service-package": 20.62,
82+
"@objectstack/service-queue": 46.31,
83+
"@objectstack/service-realtime": 17.52,
84+
"@objectstack/service-settings": 73.41,
85+
"@objectstack/service-sms": 20.24,
86+
"@objectstack/service-storage": 85.72,
87+
"@objectstack/spec": 1391.38,
88+
"@objectstack/trigger-api": 2.47,
89+
"@objectstack/trigger-record-change": 36.26,
90+
"@objectstack/trigger-schedule": 29.01,
91+
"@objectstack/types": 21.2,
92+
"@objectstack/verify": 81.38,
93+
"@objectstack/vitest-filter-preflight": 22.23,
94+
"create-objectstack": 22.85
8895
},
89-
"skippedAsCached": []
96+
"skippedAsCached": [],
97+
"skippedIncompleteSlices": [],
98+
"carriedOver": []
9099
}

0 commit comments

Comments
 (0)