ci: path-filter heavy workflows + build-once/fan-out runtime checks#2852
Conversation
…kip runtime CI The monorepo now carries website/, sdk/, and vendored frontier; before this, any commit triggered try-runtime (46m), validate-benchmarks (42m), the mainnet clone-upgrade test, docker builds, and every zombienet/e2e suite. Two patterns, chosen per workflow by whether it holds required merge checks: - Not required (try-runtime, validate-benchmarks, docker build, btcli/sdk e2e): workflow-level `paths:` filters. - Required (check-rust, eco-tests, clone-upgrade + docs-website, typescript-e2e): the workflow always triggers and a `changes` job (dorny/paths-filter, pinned) lets jobs skip themselves — a `skipped` conclusion satisfies branch protection, whereas a path-filtered workflow that never triggers leaves required checks stuck "Expected" and blocks the merge. Co-authored-by: Cursor <cursoragent@cursor.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| - "Dockerfile-localnet" | ||
| # The SDK/CLI code under test. | ||
| - "sdk/**" | ||
| - ".github/workflows/check-bittensor-e2e-tests.yml" |
There was a problem hiding this comment.
[HIGH] Executed scripts bypass the e2e workflow filter
This workflow later executes scripts/install_build_env.sh, scripts/localnet_patch.sh, and scripts/localnet.sh on a persistent self-hosted runner, but none matches this filter. A PR changing only one of those scripts will skip this workflow; the merged script can then execute during a later, unrelated nucleus-approved run. Add all directly executed scripts to the path set.
| - ".github/workflows/check-bittensor-e2e-tests.yml" | |
| - "scripts/install_build_env.sh" | |
| - "scripts/localnet_patch.sh" | |
| - "scripts/localnet.sh" | |
| - ".github/workflows/check-bittensor-e2e-tests.yml" |
| - "rust-toolchain.toml" | ||
| # Workflow definition and helper scripts used by this run. | ||
| - ".github/workflows/run-benchmarks.yml" | ||
| - ".github/scripts/**" |
There was a problem hiding this comment.
[HIGH] Benchmark runner script bypasses the workflow filter
The job directly executes scripts/benchmark_action.sh on the self-hosted Benchmarking runner, while this filter covers only .github/scripts/**. A PR modifying that executable alone skips validation and leaves it to run on a later matching PR. Include the script itself in the trigger paths.
| - ".github/scripts/**" | |
| - ".github/scripts/**" | |
| - "scripts/benchmark_action.sh" |
🛡️ AI Review — Skeptic (security review)VERDICT: VULNERABLE MEDIUM scrutiny: write-permission collaborator with substantial activity, a ~4-month-old account, no detected Gittensor association, and an explicitly justified stacked branch ci/path-filter-heavy-workflows → bittensor-core-exploration. The previously reported delayed-execution path remains: this workflow executes repository scripts on self-hosted runners but does not trigger when those scripts alone change. Findings
Prior-comment reconciliation
ConclusionThe e2e filter still lets changes to executed scripts evade immediate validation and run later when another change triggers the workflow. Add 📜 Previous run (superseded)
# 🔍 AI Review — Auditor (domain review) has not yet run on this PR. |
|
🔄 AI review updated — Skeptic: VULNERABLE |
…clone-upgrade into runtime-checks.yml One build job compiles the try-runtime wasm (production profile) and the release node binary; consumers download artifacts instead of rebuilding: - try-runtime devnet/testnet/mainnet: no Rust toolchain at all — prebuilt try-runtime-cli + the shared wasm blob. Previously three identical production builds on three runners (the mainnet job alone was ~46m). - spec_version newer than mainnet: drops from ~8m of compiling (cargo install substrate-spec-version + cargo run subtensor-tools) to seconds on ubuntu-latest — one state_getRuntimeVersion RPC call vs the spec_version literal in runtime/src/lib.rs (grep enforces exactly one match so a feature-gated second literal fails loudly). - Sudo-upgrade mainnet clone and test: consumes the node binary + release wasm artifact at the exact paths start-local-clone.sh and update-runtime-with-alice.ts expect. Consolidation into one file is forced by artifact scoping (artifacts are per-workflow-run). Required checks (clone-upgrade, docs-website) keep their names and the always-trigger + job-level-skip pattern. The now-unused .github/actions/try-runtime composite action is removed with its only caller. Co-authored-by: Cursor <cursoragent@cursor.com>
unarbos
changed the title
| - "Dockerfile-localnet" | ||
| # The SDK/CLI code under test. | ||
| - "sdk/**" | ||
| - ".github/workflows/check-bittensor-e2e-tests.yml" |
There was a problem hiding this comment.
[HIGH] Executed scripts still bypass the e2e workflow filter
This workflow checks out the PR and executes scripts/install_build_env.sh, scripts/localnet_patch.sh, and scripts/localnet.sh on self-hosted runners, but scripts/** is absent from the trigger paths. A script-only PR can merge without this workflow running, leaving its payload to execute later when an unrelated matching change triggers a nucleus-approved run. Include the executed scripts in the filter.
| - ".github/workflows/check-bittensor-e2e-tests.yml" | |
| - "scripts/**" | |
| - ".github/workflows/check-bittensor-e2e-tests.yml" |
|
🔄 AI review updated — Skeptic: VULNERABLE |
…anged pallets The skip check previously ran on the persistent Benchmarking machine after checkout, so even skipped runs occupied the one reference-hardware runner. Replaced with a decide job on ubuntu-latest (seconds, free): - Path-gating does most of the work: website/sdk/docs/CI-only pushes never touch the Benchmarking runner, while pallets/runtime/vendor/deps changes still benchmark automatically — no forgotten-label drift hole. - Pallet-only diffs benchmark just the changed pallets: the decide job emits PALLET_DIRS and benchmark_action.sh filters its discovered pallet set (exits 0 when the changed pallets have no registered benchmarks). Diffs reaching runtime/, vendor/, or dependencies still run the full suite. - Labels: run-benchmarks forces a full run; skip-validate-benchmarks still skips (now read from the event payload — check-skip-label.sh deleted with its only caller). - Nightly cron reruns the full suite on the default branch as the drift backstop, reusing the existing bench-patch artifact machinery. validate-benchmarks is not a required merge check today; the decide-job shape stays safe if that ever changes (workflow always triggers, skipped job satisfies branch protection). Co-authored-by: Cursor <cursoragent@cursor.com>
| - "Dockerfile-localnet" | ||
| # The SDK/CLI code under test. | ||
| - "sdk/**" | ||
| - ".github/workflows/check-bittensor-e2e-tests.yml" |
There was a problem hiding this comment.
[HIGH] Executed scripts still bypass the e2e workflow filter
This workflow later executes scripts/install_build_env.sh, scripts/localnet_patch.sh, and scripts/localnet.sh on self-hosted runners, but scripts/** is absent from the trigger. A script-only PR can therefore merge without this workflow running, leaving the changed script to execute during a later nucleus-approved run whose unrelated diff matches this filter. Include every checked-out executable input in the path set.
| - ".github/workflows/check-bittensor-e2e-tests.yml" | |
| - "scripts/**" | |
| - ".github/workflows/check-bittensor-e2e-tests.yml" |
|
🔄 AI review updated — Skeptic: VULNERABLE |
1 participant
Summary
try-runtime,run-benchmarks,check-docker,check-bittensor-e2e-tests): plain workflow-levelpaths:filters.check-rust,eco-tests,check-clone-upgradeincl. docs-website,typescript-e2e): the workflow always triggers; achangesjob (dorny/paths-filter v3.0.2, SHA-pinned) lets the expensive jobs skip themselves. Askippedconclusion satisfies branch protection, whereas a path-filtered workflow that never triggers leaves its required checks stuck "Expected" and blocks the merge — each file carries a comment warning about this trap.typescript-formatting(required, ~50s) still runs unconditionally; only the node builds and zombienet suites behind it are gated.vendor/**(vendored frontier compiles into the node and is fmt-checked) and the two Rust sdk crates that are workspace members.Targets
bittensor-core-explorationso PR #2846's in-flight CI run is not restarted; merge this after #2846 lands (or fold it in on the next push if #2846 needs another spin anyway).Test plan
actionlintclean on all 8 edited workflows (only pre-existing shellcheck style notes in untouched scripts)Made with Cursor
Update: build once, fan out (second commit)
try-runtime.yml,check-spec-version.yml, andcheck-clone-upgrade.ymlare consolidated into a newruntime-checks.yml(artifacts are scoped to a single workflow run, so the fan-out jobs must share a file):build runtime artifactsjob produces the try-runtime wasm (production profile +try-runtimefeature) and the release node binary + wbuild wasm.try-runtime <network>jobs no longer compile anything — prebuilttry-runtime-cli+ the wasm artifact. The endpoint genesis-verification step is preserved verbatim.spec_version newer than mainnetmoves toubuntu-latestwith zero compilation: mainnet via onestate_getRuntimeVersionRPC, local via thespec_version:literal inruntime/src/lib.rs(grep enforces exactly one match, so a feature-gated second literal fails loudly instead of silently picking one). Verified locally: RPC returns 424, grep finds exactly one literal (428).Sudo-upgrade mainnet clone and test(required check, name unchanged) downloads the node + wasm artifact at the exact pathsstart-local-clone.shandupdate-runtime-with-alice.tsexpect; all test steps unchanged.Docs and website build(required check, name unchanged) carried over as-is..github/actions/try-runtimecomposite action is deleted with its only caller.Expected effect: 3 fewer production-profile runtime builds and 2 fewer ad-hoc compiles per runtime PR; the try-runtime network jobs shrink to their state-scrape time; heavy/benchmark runner queueing drops accordingly.
Update: benchmark gate moved off the Benchmarking runner (third commit)
decidejob onubuntu-latestgatesvalidate-benchmarks: the skip/label check no longer occupies the persistent Benchmarking machine (previouslycheck-skip-label.shran there, after checkout).PALLET_DIRS(plumbed intobenchmark_action.sh, which exits 0 if the changed pallets have no registered benchmarks). Anything touchingruntime/,vendor/, or dependency files still runs the full suite.run-benchmarks= force full run;skip-validate-benchmarks= skip (unchanged semantics, now read from the event payload; the helper script is deleted with its only caller).