bench(zero-cache): add rm vs load benchmark harness - #2
Closed
Karavil wants to merge 495 commits into
Closed
Conversation
…ocicorp#5776) If the user has OTEL tracing set up in their API server then this will allow us to see the trace across both systems. zero-cache and their API server.
Swaps to the binary protocol for copying from PG rather than using the text protocol. ``` ┌──────────────┬──────────────────┬───────────────────────────┬─────────┐ │ │ Main (text COPY) │ This branch (binary COPY) │ Speedup │ ├──────────────┼──────────────────┼───────────────────────────┼─────────┤ │ Total time │ 19.45s │ 9.86s │ 1.97x │ ├──────────────┼──────────────────┼───────────────────────────┼─────────┤ │ Rate │ 142.0K rows/s │ 280.0K rows/s │ 1.97x │ ├──────────────┼──────────────────┼───────────────────────────┼─────────┤ │ Replica size │ 1256.9 MB │ 1256.9 MB │ same │ └──────────────┴──────────────────┴───────────────────────────┴─────────┘ ```
**Users MUST start validating the `X-User-ID` header in their API endpoints against the provided cookie/token auth. A missing value means logged out.** They also should stop using the `anon` sentinel in `ZeroOptions`. ### Stack 1. rocicorp#5766 - auth maintenance config knobs 2. rocicorp#5767 - zero-cache runtime refactor (this is where the state machine is introduced) 3. rocicorp#5768 - zero-cache test and harness updates 4. rocicorp#5769 - zero-client logged-out `userID` changes 5. rocicorp#5770 - zbugs adoption 6. rocicorp#5772 - clean up auth 7. rocicorp#5773 - further clean up for bugs and types 8. rocicorp#5777 - require user ID to match validated client group ### Overall model - Auth is now tracked **per websocket connection**, not as one shared auth object for the whole client group. - Each connection carries its own auth token, `userID`, and fetch context for `/query` and `/push` (URL, forwarded headers, cookies, origin). - Query and push both get a header sent to them, `X-User-ID`, which they must validate against their own auth. - The client group is pinned to a single `userID`, which is validated against the API. ### View-syncer query flow 1. A new connection is registered as **provisional**, using a state machine shared across all workers in zero-cache. 2. `initConnection` validates that connection before it is allowed to drive shared query work. 3. If validation succeeds, the connection becomes **validated**. 4. If auth changes, only that connection is demoted, revalidated, and its queries are retransformed. 5. Connection-triggered query work uses the **originating connection's** auth snapshot. 8. Shared/background query work uses one selected **validated background connection**. ### Query auth behavior - **Custom queries** are revalidated and retransformed on connect and auth changes, so query expansion always reflects the current credential. - Opaque tokens are forwarded to the API server for `/query` and `/push`, but the API server remains the real authorization boundary. ### Failure handling and maintenance - `zero-cache` can periodically revalidate validated connections and periodically rerun shared background query retransform work, using `ZERO_AUTH_REVALIDATE_INTERVAL_SECONDS` and `ZERO_AUTH_RETRANSFORM_INTERVAL_SECONDS`. - Auth failures disconnect only the failing connection. - If the selected background connection fails auth, view-syncer promotes another validated connection and retries. - Auth revision is tracked so that no race conditions exist with auth validation for two competing auth updates. ### Client and app updates - Logged-out `Zero` clients now omit `userID` on the wire instead of sending `'anon'` or an empty string. - zero-client still uses a private storage-only sentinel for IDB naming so logged-out local state remains stable. - zbugs now uses personalized rows derived from authenticated query context (`auth.sub`). - zbugs server routes now require `X-User-ID` to match the JWT on server-driven authenticated requests.
The implicit size of a postgres.js pool is 10. The reaper and change source probably don't hit that, but we intentionally cap to create that requirement to reason about max connections.
…5787) Log long transactions at a level higher than DEBUG for general debugging of long transactions that may result in replication lag alerts.
Tracking the rate of replicated changes will be useful for diagnosing replication lag, by delineating the scenario of being wedged vs being busy. Also, in combination with the existing transaction counter, the two metrics can also be used to distinguish between business from organic traffic, vs those from heavy migration-type transactions.
…icorp#5791) The BTree comparator call site must stay monomorphic across all indexes with different sort orderings. Each distinct arrow function expression gets its own SharedFunctionInfo (SFI); when multiple BTrees (e.g. primary index on [id] and secondary on [createdAt, id]) are used in the same process, the call site sees multiple SFIs and goes polymorphic or megamorphic, triggering V8 IC deopt. An earlier iteration of this change returned separate closures for the len=1 and len=2 cases (5+ distinct SFIs), which caused exactly this deopt and negated the gains. Splitting into just 2 SFIs (len=1 vs len>=2) was also tested and still measurably regresses push/edit by ~5% vs main. Fix: single return body shared across all sort configurations. Pre-extract the first two keys/directions into closed-over variables (k0/a0, k1/a1) to avoid per-call array access for the common 1- and 2-key cases. For >=3 keys the loop reads sort[i] directly — equivalent cost, no extra allocation. Benchmarks (MemorySource, 1000 rows, 2+ runs averaged, branch vs main): push add/remove, sort 1 key: 2.82 µs vs 3.17 µs (+11%) push add/remove, sort 2 keys: 2.73 µs vs 3.12 µs (+13%) push add/remove, sort 4 keys: 2.77 µs vs 3.14 µs (+12%) push edit, sort 1 key: 3.90 µs vs 4.41 µs (+12%) push edit, sort 2 keys: 3.86 µs vs 4.45 µs (+13%) push edit, sort 4 keys: 3.81 µs vs 4.47 µs (+15%) fetch (all sort lengths): neutral (scan is memory-bound) Also adds explicit 1/2/4-key sort variants to the memory-ivm-deopt benchmark to catch sort-length-specific regressions.
…rget postgres ops (rocicorp#5756) This PR adds missing `.catch(() => {})` handlers to background database operations (`warmupConnections` in `change-streamer.ts` and `disableStatementTimeout` in `pg.ts`).
Seems like there was a typo in `validatePublications`.
…-graceful shutdowns (rocicorp#5793) Distinguish between intentional, protocol-driven shutdowns such as auto-reset or incompatible-replica-version, vs unexpected, error-driven shutdowns, by using `SIGQUIT` for the former and `SIGABRT` for the latter. The behavior is the same in that drains are skipped; the only difference is in the logging level for classification/alerting purposes. The behavior for `SIGTERM` and `SIGINT` remain the same. Context: * https://rocicorp.slack.com/archives/C0ARW28F8KU/p1775749914957699
## Add zero-cache metrics Adds five new OTel metrics to improve visibility into sync worker health: 1. **`zero.sync.active-client-groups`** (Observable Gauge) — Number of active `ViewSyncerService` instances in a syncer worker. Complements the existing anonymous telemetry gauge by routing through the standard OTel/Prometheus pipeline. 2. **`zero.sync.queries`** (Observable Gauge) — Active IVM pipelines across all client groups in a syncer worker. 3. **`zero.sync.rows`** (Observable Gauge) — CVR-tracked rows across all client groups in a syncer worker. 4. **`zero.sync.lock-wait-time`** (Histogram, seconds) — Time spent waiting to acquire the `ViewSyncerService` lock per operation. 5. **`zero.sync.pipeline-resets`** (Counter) — Count of pipeline resets with a `reason` attribute distinguishing: `advancement-timeout`, `scalar-subquery`, `schema-change`, `truncation`, `permissions-change`. Per-worker averages (e.g. queries/client-group) can be derived in Grafana by dividing by `active-client-groups`. Cluster-wide aggregation is done by summing across instances. Also adds a typed `ResetPipelinesReason` to `ResetPipelinesSignal` so reset causes are tracked structurally rather than by parsing error messages.
…the backfill watermark (rocicorp#5796)
…en started (rocicorp#5803) Server startup may take arbitrarily long (e.g. a large litestream restore), and the zero-cache should respond to drain signals when starting up. This reduces the chance of more servers running than what some of the code is built to assume. Context: * https://rocicorp.slack.com/archives/C0AK5KX8HGE/p1775852012306929
# refactor(zql): Change from object union to monomorphic tuple
```ts
{type: 'add', node: Node}
{type: 'remove', node: Node}
{type: 'child', node: Node, child: ChildData}
{type: 'edit', node: Node, oldNode: Node}
```
to a monomorphic 3-tuple:
```ts
[ChangeType.Add /* SMI */, Node, null]
[ChangeType.Remove /* SMI */, Node, null]
[ChangeType.Child /* SMI */, Node, ChildData]
[ChangeType.Edit /* SMI */, Node, Node]
```
Motivation
---
The previous object union was **polymorphic** — each variant has a
different shape, so V8 must track multiple hidden classes at every
callsite that touches a `Change`. Polymorphic IC (inline cache) misses
are expensive in hot IVM push paths.
The tuple form is **monomorphic**: every `Change` is an array of the
same length with the same element types at each index, so V8 can compile
a single fast path.
Switching the discriminant from a string to an SMI also helps: SMIs are
stack-allocated and compared by value; strings are heap-allocated and
compared by hash.
Performance
---
5×5 runs on `main` vs this branch, `NODE_OPTIONS=--expose-gc` +
`inner_gc: true`, metric: median-of-5 `min`.
| Benchmark | main | branch | Δ |
|---|--:|--:|--:|
| **IVM push / edit** | | | |
| zql: edit for limited query, inside bound | 7.8 µs | 6.4 µs | **−18%**
|
| MemorySource push: edit 1 000 rows, sort 4 keys | 13.7 µs | 11.2 µs |
**−18%** |
| push: add issue (no join) | 33.3 µs | 27.5 µs | **−18%** |
| Filter: push add open issue (passes filter) | 7.5 µs | 6.8 µs |
**−9%** |
| Join: push edit issue title | 12.2 µs | 10.4 µs | **−15%** |
| Join: push add/remove issue with owner | 18.5 µs | 17.9 µs | **−3%** |
| zql: push into limited query, inside bound | 131 µs | 115 µs |
**−13%** |
| **Fetch / hydration** | | | |
| Join: fetch 1 000 issues → 20 users | 2.47 ms | 2.07 ms | **−16%** |
| MemorySource fetch: scan 1 000 rows, sort 1 key | 390 µs | 336 µs |
**−14%** |
| MemorySource fetch: scan 1 000 rows, sort 4 keys | 379 µs | 323 µs |
**−15%** |
| hydrate: issues with creator + comments | 4.51 ms | 3.96 ms | **−12%**
|
| hydrate: issues limit 50 | 250 µs | 221 µs | **−12%** |
| **Full query (Chinook)** | | | |
| tracks with artist name (not flipped) | 111.6 ms | 98.0 ms | **−12%**
|
| zqlite: all playlists | 1 093 ms | 996 ms | **−9%** |
…plication-manager handoff (rocicorp#5805) Use a Postgres row lock on the Change DB to prevent change-log cleanup during a replication-manager handoff. The previous reliance on the `/snapshot` websocket connection is susceptible to "dropping the baton" if the outgoing replication-manager is shutdown (and restarted), making it possible for an RM to purge change-log entries needed by the incoming RM: * https://rocicorp.slack.com/archives/C0AS00SDCKV/p1775849670636429 Instead of using the `/snapshot` protocol (which also could only be best-effort for replication-manager startup, since a previous replication-manager may not exist), a share lock on the earliest row of the "changeLog" table is used to prevent purges. At startup, the replication-manager: 1. acquires a "FOR SHARE" lock on the earliest `changeLog` row 2. restores from litestream if such a row exists (otherwise, initiates initial sync). 3. verifies that the restored backup matches the constraints read from the Change DB a. retrying the restore if not 4. assumes Change DB ownership (as before) 5. releases the "FOR SHARE" lock The "FOR SHARE" lock prevents any change-log DELETE's from happening until the lock is released. By waiting until ownership is assumed (a write to the ownership column) before releasing the lock, this mechanism is _backwards compatible_ with previous change-streamer code, which checks the ownership column before committing a transaction purge, and aborts if ownership has changed. As an optimization, new change-streamer code will first attempt to acquire a "FOR UPDATE NOWAIT" lock on the first row to abort early (and release transaction resources) if the row is locked. An additional benefit of this protocol is that it serves to verify the expected constraints of the backup at replication-manager startup; the view-syncer receives these constraints via the `/snapshot` protocol, and now the replication-manager determines the equivalent information directly from the change-db. The associated verification of constraints allows the existing delete-and-retry loop to handle pathological cases in which the wrong backup is restored, all happening before the server announces readiness. Theoretically, the view-syncer logic _could_ be switched to this protocol as well. However: * having view-syncers directly access the Change DB breaks the replication-manager abstraction and makes the system harder to maintain and upgrade. In particular, this would complicate the migration to an upcoming architecture in which Postgres is no longer used for storing the change-log. * the consequence of a "dropped baton" in the case of a view-syncer startup is much less disruptive, as it involves a restart and restore rather than a full replica resync, and view-syncers are redundant as well. As such, view-syncer restores continue to use the existing `/snapshot` protocol for reserving snapshots.
When a purge-lock is held, let the lock error propagate instead of catching and returning 0. The calling code will then retry that watermark instead of considering it purged. In practice this doesn't really matter since a locked row means that the old replication-manager will be taken over, but the semantics are more correct to throw an exception from the function as opposed to returning 0.
…#5807) Stop alphabetically sorting primaryKey, uniqueKeys, and allPotentialPrimaryKeys in computeZqlSpecs. The sorting caused compound primary key columns to be reordered alphabetically (e.g. [callId, userId, connectionId] became [callId, connectionId, userId]), which resulted in incorrect ORDER BY clauses and index column ordering in SQLite. This mismatch between the ORDER BY column order and the actual index column order forces SQLite to fall back to full table scans instead of using the index, causing severe performance degradation. The sorting was originally added as an optimization so that normalizedKeyOrder() in row-key.ts could take its fast path (already-sorted check) and avoid allocating new objects. However, these two concerns are independent: - Row identity (normalizedKeyOrder) sorts RowKey object properties alphabetically at each point of use (change-log, snapshotter, CVR) for deterministic stringification. It handles unsorted input fine. - Query ordering (completeOrdering -> addPrimaryKeys) appends PK columns to ORDER BY in the PK array order from the tableSpec. This is where the bug manifested. Removing the pre-sort is safe: row identity still gets normalized at point of use, and query ordering now preserves the index-defined column order. Fixes: https://bugs.rocicorp.dev/p/zero/issue/246641
Get rid of one allocation in every join push
Pre-commit: auto-format staged files with oxfmt. Pre-push: run syncpack lint, oxlint, and oxfmt check in parallel on changed files. Uses simple-git-hooks and zx.
…rsion overrides (rocicorp#6034) Enhance the pnpm workspace configuration by adding minimum release age exclusions and specifying version overrides for several packages. This improves dependency management and ensures compatibility with specific package versions.
.one() sets limit: 1 in the AST and singular: true in the format. .limit(1) sets only limit: 1. Previously both produced the same hash, causing them to share a view and return the wrong shape. Add tests covering .one() vs .limit(1) and queries that differ only in their relationship. https://discord.com/channels/830183651022471199/1288232858795769917/1506476345662636092
``` ┌──────────┬────────┬─────────────────────┬─────┐ │ │ Before │ After commits #1+#2 │ Δ │ ├──────────┼────────┼─────────────────────┼─────┤ │ Critical │ 5 │ 2 │ -3 │ ├──────────┼────────┼─────────────────────┼─────┤ │ High │ 67 │ 17 │ -50 │ ├──────────┼────────┼─────────────────────┼─────┤ │ Medium │ 64 │ 26 │ -38 │ ├──────────┼────────┼─────────────────────┼─────┤ │ Low │ 11 │ 1 │ -10 │ └──────────┴────────┴─────────────────────┴─────┘ ``` --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…orp#6039) setVersionInWorkspace previously rewrote @rocicorp/zero in apps/zbugs and apps/zql-viz to point at the new canary version (replacing workspace:*). That mutation was load-bearing for nothing: neither app is published from this script, and workspace:* already resolves to the bumped local packages/zero version via pnpm's linkWorkspacePackages. The mutation had two costs: * It diverged the lockfile from package.json on every release, which caused the canary CI dry-run to fail under pnpm's CI-default --frozen-lockfile (run 26179682012). * It changed apps/* manifests in the temp clone for no observable benefit, making the release commit noisier than it needed to be. After this change, setVersionInWorkspace only touches packages/zero/package.json's version field, the lockfile stays in sync, and the second pnpm install runs cleanly under CI's frozen default. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removing deploy scripts entirely.
…ocicorp#6043) Since the pnpm migration (rocicorp#5997), Rollup's preserveModules emitted transitive chunks under out/packages/<pkg>/... instead of out/<pkg>/..., because under pnpm's isolated linker the resolved module paths run through .pnpm/ symlinks and Rollup's implicit common-ancestor root ends up at mono/ rather than mono/packages/. This split .js chunks away from their sibling .d.ts files (tsc still emits to the flat tree) and broke imports like @rocicorp/zero/server/adapters/pg at runtime. Pinning preserveModulesRoot to packages/ makes chunk paths install-state independent and restores the layout shipped through 1.5.0.
This was flagged by e18e lint after upgrade.
Modern nodejs can run typescript.
…rp#6046) Adapter entries (./server/adapters/pg, ./react, ./solid) imported pg, react, and solid-js, but they weren't in dependencies or peerDependencies, so Rolldown followed the imports and inlined them (along with pg-pool, pg-protocol, pg-types, postgres-*, split2, xtend, etc.) into out/node_modules/.pnpm/. Consumers ended up with duplicate copies that break Pool instanceof checks and React's single-instance rule. Declare pg, drizzle-orm, react, and solid-js as optional peerDependencies so getExternalFromPackageJSON marks them external. Add @opentelemetry/semantic-conventions to dependencies — it's used directly by zero-cache and is an internal runtime dep, not a peer. Add a post-build assertion that fails if out/node_modules exists, so a future undeclared import that gets inlined breaks the build with a pointer at the fix instead of silently shipping in the tarball.
Is replication lag high or did the replication stream just stop entirely? Adds a fourth replication-lag gauge that reports only the measured round-trip from the most recently received lag report, without the watchdog `max(..., now - nextSendTimeMs)` component of total_lag. When the report stream stalls, total_lag grows linearly while last_total_lag stays flat at the last real measurement — comparing the two distinguishes actual replication lag from a stuck change-source connection. This incident was the impetus for this: https://rocicorp.slack.com/archives/C013XFG80JC/p1779331755395539 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…6040) `npm install -g` of zero's tarball runs install/postinstall scripts for every transitive dep. Switch to `pnpm add` with `pnpm.onlyBuiltDependencies` in /opt/app/package.json; pnpm v10+ ignores dependency lifecycle scripts by default and only runs them for allowlisted packages. Allowlist: `@rocicorp/zero-sqlite3` (its `install` runs prebuild-install to fetch the native binary, which isn't in its tarball) and `esbuild` (via tsx; included defensively until runtime confirms tsx is unused). Install is local to /opt/app with `node-linker=hoisted` so the existing zero-sqlite3 symlink and config.yml copy paths still resolve; PATH is extended so CMD drops `npx`. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
re: rocicorp#6040 (comment) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds https://docs.zizmor.sh/ to do automatic security analysis of CI workflows. Fixes coming in next PR.
## Summary I don't think this is used anymore but alas it's being updated just in case. - Add explicit permissions, job name, and concurrency to the Replicache release asset workflow. - Pin all workflow actions to commit SHAs. - Disable persisted checkout credentials. - Disable setup-node package-manager caching in the release artifact workflow. - Replace `softprops/action-gh-release` with the built-in `gh release` CLI for release asset upload/create behavior. ## Original zizmor findings - `packages/replicache/.github/workflows/upload-release-assets.yml`: `artipacked` because checkout did not set `persist-credentials: false`. - `packages/replicache/.github/workflows/upload-release-assets.yml`: `excessive-permissions` due to missing explicit `permissions`. - `packages/replicache/.github/workflows/upload-release-assets.yml`: `unpinned-uses` for tag-pinned `actions/checkout`, `pnpm/action-setup`, `actions/setup-node`, and `softprops/action-gh-release`. - `packages/replicache/.github/workflows/upload-release-assets.yml`: `cache-poisoning` because a tag-triggered release artifact workflow enabled setup-node package-manager caching. - `packages/replicache/.github/workflows/upload-release-assets.yml`: `anonymous-definition` because the job had no `name`. - `packages/replicache/.github/workflows/upload-release-assets.yml`: `concurrency-limits` due to missing workflow concurrency. - `packages/replicache/.github/workflows/upload-release-assets.yml`: `superfluous-actions` because `zizmor` recommends using `gh release` instead of `softprops/action-gh-release`. ## Verification - `zizmor --no-progress --color=never --persona=auditor packages/replicache/.github/workflows/upload-release-assets.yml`
## Summary - Add explicit permissions and missing concurrency to benchmark-related workflows. - Document Bencher and github-action-benchmark write permissions. - Keep intentional self-hosted benchmark runners with targeted `zizmor` ignores and explanations. - Move Bencher upload jobs to the `Benchmark` environment. - Stop passing `BENCHER_API_TOKEN` through reusable workflow callers; the reusable upload jobs now read it from the `Benchmark` environment. ## Original zizmor findings - `.github/workflows/bencher-benchmarks-pr.yml`: `excessive-permissions`, `self-hosted-runner`, `undocumented-permissions`, and `concurrency-limits`. - `.github/workflows/bencher-benchmarks.yml`: `excessive-permissions`, `self-hosted-runner`, `undocumented-permissions`, and `concurrency-limits`. - `.github/workflows/bencher-file-sizes-pr.yml`: `excessive-permissions` and `undocumented-permissions`. - `.github/workflows/bencher-file-sizes.yml`: `excessive-permissions` and `undocumented-permissions`. - `.github/workflows/bundle-sizes.js.yml`: `concurrency-limits` and `undocumented-permissions` for `contents: write`. - `.github/workflows/perf-v2.js.yml`: `concurrency-limits`, two `self-hosted-runner` findings, and `undocumented-permissions` for `contents: write`. - `.github/workflows/reusable-benchmark.yml`: `excessive-permissions`, `self-hosted-runner`, and `undocumented-permissions`. - `.github/workflows/reusable-file-sizes.yml`: `excessive-permissions` and `undocumented-permissions`. ## Environment secrets - Add `BENCHER_API_TOKEN` to the `Benchmark` environment before merging. ## Verification - `zizmor --no-progress --color=never --persona=auditor .github/workflows/bencher-benchmarks-pr.yml .github/workflows/bencher-benchmarks.yml .github/workflows/bencher-file-sizes-pr.yml .github/workflows/bencher-file-sizes.yml .github/workflows/bundle-sizes.js.yml .github/workflows/perf-v2.js.yml .github/workflows/reusable-benchmark.yml .github/workflows/reusable-file-sizes.yml`
## Summary
- Remove the Playwright composite action write to `GITHUB_ENV`.
- Add explicit read-only workflow permissions for API snapshot, JS CI,
and perf smoke.
- Disable persisted checkout credentials in perf smoke.
- Move the JS test shard expression into an environment variable before
shell use.
## Original zizmor findings
- `.github/actions/playwright-install/action.yml`: `github-env` for
writing to `GITHUB_ENV`.
- `.github/workflows/api-snapshot.yml`: `excessive-permissions` due to
missing explicit `permissions`.
- `.github/workflows/js.yml`: `excessive-permissions` due to missing
explicit `permissions` across jobs.
- `.github/workflows/js.yml`: `template-injection` for direct `${{
matrix.shard }}` expansion in a `run` command.
- `.github/workflows/perf-smoke.yml`: `artipacked` because checkout did
not set `persist-credentials: false`.
- `.github/workflows/perf-smoke.yml`: `excessive-permissions` due to
missing explicit `permissions`.
## Verification
- `zizmor --no-progress --color=never --persona=auditor
.github/actions/playwright-install/action.yml
.github/workflows/api-snapshot.yml .github/workflows/js.yml
.github/workflows/perf-smoke.yml`
## Summary
- Add explicit top-level permissions and concurrency to the release
workflow.
- Scope the release job to the `Release` environment and the mirror job
to the `Mirror` environment.
- Disable persisted checkout credentials in the release workflow while
preserving git auth with process-scoped `GIT_CONFIG_*` env.
- Document release job write permissions.
- Move release PR comment template values into environment variables and
write the body through a file.
## Original zizmor findings
- `.github/workflows/release.yml`: `artipacked` because checkout did not
set `persist-credentials: false`.
- `.github/workflows/release.yml`: `excessive-permissions` due to
missing top-level `permissions: {}`.
- `.github/workflows/release.yml`: `undocumented-permissions` for
`contents: write`, `id-token: write`, and `pull-requests: write`.
- `.github/workflows/release.yml`: `concurrency-limits` due to missing
workflow concurrency.
- `.github/workflows/release.yml`: `secrets-outside-env` for
`secrets.DOCKERHUB_TOKEN`.
- `.github/workflows/release.yml`: `template-injection` for direct PR
number and release version template expansions inside `run`.
- `.github/workflows/mirror.yml`: `concurrency-limits` due to missing
workflow concurrency.
- `.github/workflows/mirror.yml`: `secrets-outside-env` for
`secrets.ROCICORP_MIRROR_APP_PRIVATE_KEY`.
## Environment secrets
- `Mirror` already has `ROCICORP_MIRROR_APP_PRIVATE_KEY`.
- Add `DOCKERHUB_TOKEN` to the `Release` environment before merging.
## Verification
- `zizmor --no-progress --color=never --persona=auditor
.github/workflows/release.yml .github/workflows/mirror.yml`
This PR removes all Bencher-related GitHub Actions workflows that were used for performance benchmarking and file size tracking. ## Summary Removed the following workflow files: - `.github/workflows/reusable-benchmark.yml` - Reusable workflow for running benchmarks - `.github/workflows/reusable-file-sizes.yml` - Reusable workflow for tracking file sizes - `.github/workflows/bencher-benchmarks.yml` - Main branch benchmark workflow - `.github/workflows/bencher-benchmarks-pr.yml` - PR benchmark workflow - `.github/workflows/bencher-file-sizes.yml` - Main branch file size workflow - `.github/workflows/bencher-file-sizes-pr.yml` - PR file size workflow ## Details These workflows orchestrated performance testing and file size monitoring via the Bencher platform, including: - Running benchmarks on self-hosted hardware for packages (replicache, shared, zero-client, zero-cache, zql-benchmarks) - Building and measuring file sizes for the zero package - Uploading results to Bencher for both main branch and PR runs - Setting up Playwright for benchmark dependencies The removal of these workflows indicates a shift away from Bencher-based performance tracking infrastructure. https://claude.ai/code/session_017bfWhxLhvAdY8BDWsUaoZN Co-authored-by: Claude <noreply@anthropic.com>
…consumers (rocicorp#6059) Motivation to break cyclic dependencies
Why: make RM to serving-replica perf work measurable and reviewable before changing production paths * add a v6 RM to serving replica benchmark with reconnect catchup * add config, scenario, and workload tests * expose package scripts for smoke and e2e runs
Owner
Author
|
Replaced by rocicorp/mono PRs rocicorp#6070 through rocicorp#6078. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
what changed
Adds the
rm-vs-loadbenchmark harness for measuring RM to serving-replica stream throughput, catchup lag, websocket traffic, apply mode, and reconnect behavior.benchmark output
Baseline run from this harness,
medium-wide-batch-pressure, 15s:validation