Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -558,6 +558,9 @@ jobs:
contents: read
env:
CARGO_TERM_COLOR: always
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand Down Expand Up @@ -586,6 +589,13 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

- name: Clippy (default features)
run: cargo clippy --workspace --all-targets --locked -- -D warnings -W clippy::dbg_macro

Expand Down Expand Up @@ -624,6 +634,9 @@ jobs:
# the source with `Box::pin` (see `helpers::cost::cost_harness`); this
# keeps CI deterministic across runners while that work continues.
RUST_MIN_STACK: 16777216
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand All @@ -645,6 +658,13 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

# GQ Logic Tests owns GQT in both build shapes, including its DST runtime;
# dst.yml owns the DST suite (excluded by name: with the workspace
# `--cfg tokio_unstable` it no longer compiles empty here).
Expand Down Expand Up @@ -753,6 +773,9 @@ jobs:
# A missing predecessor binary fails the genuine journey instead of
# skipping it: this context is required on every change.
OMNIGRAPH_REQUIRE_STORAGE_UPGRADE_TESTS: '1'
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand All @@ -773,6 +796,13 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

- name: Check storage compatibility gate configuration
run: python3 scripts/check-storage-upgrade-ci.py --self-test

Expand Down Expand Up @@ -889,6 +919,9 @@ jobs:
env:
CARGO_TERM_COLOR: always
RUST_MIN_STACK: 16777216
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand All @@ -910,6 +943,13 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

# CI pins only the immediate predecessor. Older seams remain env-gated
# and on-demand, per docs/dev/testing.md.
- name: Build genuine final-v5 CLI
Expand Down Expand Up @@ -970,6 +1010,9 @@ jobs:
contents: read
env:
CARGO_TERM_COLOR: always
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Skip when no engine input changed
if: needs.classify_changes.outputs.run_full_ci != 'true'
Expand Down Expand Up @@ -1000,6 +1043,14 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
if: needs.classify_changes.outputs.run_full_ci == 'true'
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

- name: Build omnigraph-server with aws feature
if: needs.classify_changes.outputs.run_full_ci == 'true'
run: cargo build --locked -p omnigraph-server --features aws
Expand Down Expand Up @@ -1050,6 +1101,9 @@ jobs:
OMNIGRAPH_S3_TEST_PREFIX: github-actions
CARGO_TERM_COLOR: always
RUST_MIN_STACK: 16777216
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand All @@ -1074,6 +1128,13 @@ jobs:
# A red `main` run still seeds the cache pull requests restore.
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

- name: Start RustFS
# Pinned to 1.0.0-beta.12 (2026-07-30). beta.8 carried hyper's short
# header_read_timeout for idle keep-alive connections (rustfs#4360,
Expand Down Expand Up @@ -1245,6 +1306,9 @@ jobs:
OMNIGRAPH_AZURE_TEST_CONTAINER: omnigraph-tests
CARGO_TERM_COLOR: always
RUST_MIN_STACK: 16777216
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand All @@ -1265,6 +1329,13 @@ jobs:
key: azurite-azure
save-if: ${{ github.ref == 'refs/heads/main' }}

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

- name: Start pinned Azurite Blob service
# Azurite 3.35.0, pinned to the immutable multi-arch index digest.
run: |
Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/dst.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ on:

concurrency:
group: dst-${{ github.ref }}
# Cache saves are push-only, so a superseded PR run loses nothing when
# Archive cache saves are push-only, so a superseded PR run loses nothing when
# cancelled; main runs stay alive as ci.yml's do.
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Expand Down Expand Up @@ -145,6 +145,10 @@ jobs:
timeout-minutes: 60
permissions:
contents: read
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- name: Checkout source
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
Expand All @@ -167,6 +171,13 @@ jobs:
# eviction churn; PR runs still restore from it.
save-if: ${{ github.event_name == 'push' }}

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

# The whole active suite: lib proofs (arbiter, judges, classifiers,
# census guards) + scenario pins + the torn-init pin. Instruments
# stay #[ignore]d — nightly's job.
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/gq-logic-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ jobs:
# Same floor as the Test Workspace job: unoptimized builds keep every
# nested async engine frame, deeper than a default 2 MiB thread stack.
RUST_MIN_STACK: 16777216
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
steps:
# Check the classification copies on skipped changes too.
- name: Checkout source
Expand Down Expand Up @@ -175,6 +178,14 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

# sccache fetches unchanged dependency compiles when rust-cache misses
# (docs/dev/ci.md, Full correctness graphs).
- name: Run sccache
if: needs.classify_changes.outputs.run_gqt == 'true'
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
with:
version: "v0.16.0"

# Every cargo invocation of this job selects the same three packages,
# so the engine graph resolves one feature set and builds once; a
# narrower selection rebuilds every crate whose features differ (the
Expand Down
34 changes: 31 additions & 3 deletions docs/dev/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -356,12 +356,40 @@ jobs also save when red (`cache-on-failure`): dependency artifacts are valid
whatever the test verdict, and a red seed run would otherwise leave every
pull request cold until `main` is green again.

The active PR-path Rust jobs also install `sccache` through the
commit-pinned `mozilla-actions/sccache-action`, with the binary version
pinned to `v0.16.0`. Each job sets `RUSTC_WRAPPER=sccache` and
`SCCACHE_GHA_ENABLED=true`. `SCCACHE_GHA_RW_MODE` is `READ_WRITE` on
`refs/heads/main` and `READ_ONLY` on every other ref, including pull requests,
merge-queue branches and tags. This keeps per-compilation uploads under
the same main-only policy as the archive cache; `rust-cache`'s `save-if`
does not control `sccache`. Both caches share the repository's Actions
cache quota. The vocabulary audit (nightly toolchain, disabled) and the
release and publish workflows carry no `sccache` step.

`rust-cache` restores an archive keyed on the toolchain, build environment,
manifests and `Cargo.lock`, with prefix fallback when dependencies change.
When Cargo still invokes rustc, `sccache` can reuse retained compilations
whose source, compiler, flags, paths and dependency hashes match.
`rust-cache` exports `CARGO_INCREMENTAL=0`, so eligible workspace library
compilations can be cached too. Linking remains uncached, including
binaries, test executables and proc-macro crates; unchanged source alone
does not guarantee a hit. Adding `RUSTC_WRAPPER` changes `rust-cache`'s
environment hash, including its prefix restore key, so existing archives
do not seed the first run. Changing the pinned `sccache` version also
changes its cache namespace. Cargo can print `Compiling` on a cache hit:
use the action's post-step hit/miss counts and build timings to measure
Comment thread
azimafroozeh marked this conversation as resolved.
reuse. The speedup and cache retention need hosted-run evidence.

A cache is derived state, and no job may need one to fit its budget. The
repository's caches exceed GitHub's cap, so every save evicts the least
recently used entries: within one `main` run, the jobs that finish first are
evicted first by the saves of the jobs that finish last. A run cancelled by
its timeout saves no usable cache either (issue #755: every run after the
cancelled `main` run restored nothing). Each `GQ Logic Tests` budget
its timeout does not save a `rust-cache` archive
([issue #755](https://github.com/ModernRelay/omnigraph/issues/755): every run
after the cancelled `main` run restored nothing). Completed `sccache`
uploads can survive a later cancellation, but neither cache is guaranteed.
Each `GQ Logic Tests` budget
therefore covers a cold build with margin, and a second `cargo` invocation
in a job must not change the dependency graph: it selects packages whose
graph the first invocation already built, or it rebuilds every crate whose
Expand Down Expand Up @@ -424,7 +452,7 @@ list):

- **`dst.yml`** (per PR and `main` push that changes engine input, through
its own `Classify Changes (DST)` copy; a superseded PR run is cancelled,
which loses nothing because its cache saves are push-only): the pinned
which loses no cache uploads because PRs write neither cache): the pinned
deterministic suite — every failure line carries the universe seed, so a
red run is reproducible locally from the log alone. The job also lints the shipped
engine shape (`-p`, no `dst` feature), which workspace feature
Expand Down
Loading