From 74d71df66e58569686ae99ca908fcae670dd987f Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 10:06:00 +0000 Subject: [PATCH 01/21] Add configurable cache size and per-derivation cache stats Expose the cache-nix-action garbage-collection threshold as a new `max-cached-store-size` input (default 8G, up from a hardcoded 2G). The old 2G ceiling triggered a store GC before every cache save that evicted freshly-built outputs - including unrooted `nix flake check` results - so they were rebuilt on every run. A generous default keeps them warm; the value maps to the uncompressed store size (what the GC measures), and can be set to an empty string to disable collection entirely. Also report a per-derivation cache breakdown to the job summary (pondinfra#23): how many store paths were restored from the GitHub Actions cache, substituted from upstream binary caches, or built locally on the runner, with locally-built paths listed individually when there are fewer than 100. Built-vs-substituted is determined exactly from Nix's own `ultimate` flag, comparing store snapshots taken before/after the cache restore and after the build. The post-build report runs in the job's post phase via pyTooling/Actions/with-post-step, since composite actions can't declare a post step of their own. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- README.md | 59 ++++++++++++++++- action.yml | 29 ++++++++- scripts/cache-stats-report.sh | 112 ++++++++++++++++++++++++++++++++ scripts/cache-stats-snapshot.sh | 47 ++++++++++++++ 4 files changed, 244 insertions(+), 3 deletions(-) create mode 100755 scripts/cache-stats-report.sh create mode 100755 scripts/cache-stats-snapshot.sh diff --git a/README.md b/README.md index dd1b8fc..0bb3187 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,10 @@ single drop-in action. ## Features - Installing Nix using [cachix/install-nix-action](https://github.com/cachix/install-nix-action) -- Caching Nix derivations using [nix-community/cache-nix-action](https://github.com/nix-community/cache-nix-action) +- Caching Nix derivations using [nix-community/cache-nix-action](https://github.com/nix-community/cache-nix-action), + with a tunable cache size that keeps build outputs warm across runs +- Reporting per-derivation cache stats to the GitHub Actions job summary — how many store paths + were restored from the GitHub cache, substituted from upstream caches, or built locally - Automagically setting up environments from `.envrc` using direnv - Commenting with [mdarocha/comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog) when a PR updates `flake.lock` - Freeing up runner disk space before installing Nix using [wimpysworld/nothing-but-nix](https://github.com/wimpysworld/nothing-but-nix) @@ -49,6 +52,7 @@ jobs: |--------------------|-------------------------------------------------------------------------------------------------------|----------------------| | `token` | Github authentication token to use | `${{ github.token }}` | | `free-up-all-storage` | Aggressively free up all possible disk space on the runner before installing Nix, using [wimpysworld/nothing-but-nix](https://github.com/wimpysworld/nothing-but-nix) | `false` | +| `max-cached-store-size` | Maximum (uncompressed) Nix store size to keep in the cache, e.g. `8G`, `512M`, or a plain byte count. Set to an empty string to disable garbage collection and cache the whole store. See [Cache size](#cache-size). | `8G` | ### Freeing up storage @@ -68,6 +72,57 @@ and is skipped gracefully on other platforms. free-up-all-storage: true ``` +### Cache size + +The Nix store is cached with [nix-community/cache-nix-action](https://github.com/nix-community/cache-nix-action). +Just before a new cache is saved, the action garbage-collects old store paths until the store is at or +below `max-cached-store-size` (default `8G`), so the cache doesn't grow without bound. + +A few things worth knowing when tuning this: + +- **The limit is on the uncompressed store.** That's what the underlying garbage collector measures + (`nix store gc --max`). The cache uploaded to GitHub is compressed and in practice ends up roughly + 2–4x smaller (e.g. a ~5.5 GiB store compresses to under ~2 GiB). There is no way to cap the + compressed size directly, because garbage collection happens before compression — use that ratio as a + rule of thumb. GitHub gives each repository 10 GiB of Actions cache and evicts least-recently-used + entries beyond that, so a store of a handful of GiB leaves plenty of headroom. +- **Set the limit above what a full build produces.** Garbage collection only runs when the store + exceeds the limit *and* the primary key didn't hit exactly. Keeping the ceiling comfortably above your + build's store size means the freshly-built paths survive collection and actually get saved (and then + restored fully warm on the next run) instead of being collected away and rebuilt every time. This is + why the default is a generous `8G` rather than something small. +- **Some commands don't create GC roots.** Notably `nix flake check` builds derivations without leaving + a GC root, so their outputs count as garbage. They're kept in the cache only if they fit under this + limit — another reason to keep it generous. +- **Disabling collection.** Set `max-cached-store-size` to an empty string to skip garbage collection + entirely and cache the whole store. Only do this if you're confident the store stays comfortably under + GitHub's cache limits. + +```yaml +- uses: mdarocha/nix-magic-setup@v1.1.0 + with: + max-cached-store-size: 6G +``` + +### Cache stats + +After your build steps have run, the action writes a breakdown to the +[job summary](https://github.blog/news-insights/product-news/supercharging-github-actions-with-job-summaries/) +showing where each Nix store path came from: + +- **♻️ Restored from GitHub Actions cache** — paths the [cache](#cache-size) restored, so they didn't + need to be fetched or built. +- **⬇️ Substituted from upstream caches** — paths pulled from binary caches (`cache.nixos.org`, Cachix, + and any `extra-substituters` from your `flake.nix`) during the build. +- **🔨 Built locally** — paths that were built on the runner because no cache had them. When there are + fewer than 100, they're listed individually so you can see exactly what wasn't cached. + +The breakdown works by snapshotting the store before and after the cache is restored, and classifying +whatever the build adds using Nix's own `ultimate` flag (set on locally-built paths). It runs in the +job's post phase — after your build — which the action arranges via +[pyTooling/Actions/with-post-step](https://github.com/pyTooling/Actions), since a composite action +can't declare a post step of its own. No configuration is required; it reports automatically. + ## Permissions required This action uses the workflows' `GITHUB_TOKEN` by default. Certain features require specific permissions to work. @@ -84,4 +139,4 @@ Certain features also only work in the context of a cloned repository, so they r In the future, this action is planned to also: - Comment on PRs with [nix-diff](https://github.com/Gabriella439/nix-diff) -- Show stats like build times, cache hits vs. misses in GitHub Actions summaries +- Show build times in GitHub Actions summaries alongside the cache stats diff --git a/action.yml b/action.yml index 18d8349..e128f15 100644 --- a/action.yml +++ b/action.yml @@ -12,6 +12,10 @@ inputs: description: "Aggressively free up all possible disk space on the runner before installing Nix, using wimpysworld/nothing-but-nix. When false, a minimal amount of space is still reclaimed" required: false default: "false" + max-cached-store-size: + description: "Maximum size of the Nix store to keep in the cache, e.g. '8G', '512M', or a plain byte count. Just before a new cache is saved, older store paths are garbage-collected until the (uncompressed) Nix store is at or below this size. The cache uploaded to GitHub is compressed and is typically 2-4x smaller than this value. Set to an empty string to disable garbage collection and cache the whole store." + required: false + default: "8G" runs: using: composite steps: @@ -29,20 +33,43 @@ runs: shell: bash run: bash "${{ github.action_path }}/scripts/setup-nixconfig.sh" + - name: Snapshot Nix store before cache restore + shell: bash + run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" pre + - name: Setup Nix Cache + id: nix-cache uses: nix-community/cache-nix-action@7df957e333c1e5da7721f60227dbba6d06080569 # v7.0.2 with: nix: true save: true primary-key: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} restore-prefixes-first-match: nix-${{ runner.os }}- - gc-max-store-size: 2G + gc-max-store-size: ${{ inputs.max-cached-store-size }} purge: true purge-prefixes: nix-${{ runner.os }}- purge-primary-key: never purge-last-accessed: "P14D" token: ${{ inputs.token }} + - name: Snapshot Nix store after cache restore + shell: bash + env: + CACHE_HIT_PRIMARY_KEY: ${{ steps.nix-cache.outputs.hit-primary-key }} + CACHE_HIT_FIRST_MATCH: ${{ steps.nix-cache.outputs.hit-first-match }} + CACHE_PRIMARY_KEY: ${{ steps.nix-cache.outputs.primary-key }} + CACHE_RESTORED_KEY: ${{ steps.nix-cache.outputs.restored-key }} + run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" post + + # Register a post-phase step that runs after the workflow's build steps, so + # it can report which store paths came from where. Composite actions can't + # declare a post step of their own, so this borrows one via with-post-step. + - name: Report cache stats after build + uses: pyTooling/Actions/with-post-step@fdcb8e6fb145f72c8e2c9c3ff6160dc62cc3e22c # v7.9.0 + with: + main: 'echo "nix-magic-setup: Nix cache stats will be reported in the post step, after the build."' + post: 'bash "${{ github.action_path }}/scripts/cache-stats-report.sh"' + - name: Check for .envrc id: check-envrc shell: bash diff --git a/scripts/cache-stats-report.sh b/scripts/cache-stats-report.sh new file mode 100755 index 0000000..730149c --- /dev/null +++ b/scripts/cache-stats-report.sh @@ -0,0 +1,112 @@ +#!/usr/bin/env bash +set -euo pipefail +export LC_ALL=C + +# Renders a per-derivation cache breakdown to the GitHub Actions job summary: +# how many store paths were restored from the GitHub Actions cache, substituted +# from upstream binary caches, or built locally during this run. Built paths are +# listed individually when there are fewer than 100 of them. +# +# Runs in the job's post phase (via pyTooling/Actions/with-post-step), after the +# workflow's build steps, so it can compare the store against the snapshots +# taken during setup. Everything here is best-effort: a failure must never fail +# the job, so the caller ignores the exit status. + +state_dir="${RUNNER_TEMP:-/tmp}/nix-magic-setup" +summary_file="${GITHUB_STEP_SUMMARY:-/dev/stdout}" + +pre_file="$state_dir/pre-paths.txt" +post_file="$state_dir/post-paths.txt" +meta_file="$state_dir/cache-meta.env" + +if [ ! -f "$post_file" ]; then + echo "nix-magic-setup: no store snapshot found, skipping cache stats." + exit 0 +fi +[ -f "$pre_file" ] || : > "$pre_file" + +workdir="$(mktemp -d)" +trap 'rm -rf "$workdir"' EXIT + +tab="$(printf '\t')" + +# Current store contents with each path's `ultimate` flag. Nix sets +# ultimate=true for paths built locally and false for paths pulled from a binary +# cache, which is exactly the built-vs-substituted distinction we need. +# `path-info --json` is an object on newer Nix and an array on older Nix; +# normalise both to `\t` lines. +if ! nix --extra-experimental-features nix-command path-info --all --json 2>/dev/null \ + | jq -r ' + (if type == "object" then to_entries | map(.value + {path: .key}) else . end) + | .[] + | select((.path | endswith(".drv")) | not) + | [.path, (if .ultimate then "built" else "sub" end)] | @tsv + ' \ + | sort -u > "$workdir/now.tsv"; then + echo "nix-magic-setup: could not query the Nix store, skipping cache stats." + exit 0 +fi +cut -f1 "$workdir/now.tsv" > "$workdir/now.txt" + +# What the GitHub Actions cache provided and is still present: (post - pre) ∩ now +comm -23 "$post_file" "$pre_file" | comm -12 - "$workdir/now.txt" > "$workdir/from-github.txt" + +# What the build added on top of the restored store: now - post +comm -23 "$workdir/now.txt" "$post_file" > "$workdir/new.txt" + +# Split the new paths into built-locally vs substituted-from-upstream using the +# ultimate flag captured above. +join -t "$tab" "$workdir/new.txt" "$workdir/now.tsv" > "$workdir/new-flagged.tsv" +awk -F'\t' '$2 == "built" { print $1 }' "$workdir/new-flagged.tsv" > "$workdir/built.txt" +awk -F'\t' '$2 == "sub" { print $1 }' "$workdir/new-flagged.tsv" > "$workdir/upstream.txt" + +github_count=$(wc -l < "$workdir/from-github.txt" | tr -d ' ') +built_count=$(wc -l < "$workdir/built.txt" | tr -d ' ') +upstream_count=$(wc -l < "$workdir/upstream.txt" | tr -d ' ') + +# Cache restore status line. +cache_line="" +if [ -f "$meta_file" ]; then + # shellcheck source=/dev/null + . "$meta_file" + if [ "${CACHE_HIT_PRIMARY_KEY:-}" = "true" ]; then + cache_line="✅ GitHub Actions cache hit on the exact key (\`${CACHE_RESTORED_KEY:-}\`)." + elif [ "${CACHE_HIT_FIRST_MATCH:-}" = "true" ]; then + cache_line="♻️ GitHub Actions cache restored from a prefix match (\`${CACHE_RESTORED_KEY:-}\`)." + else + cache_line="❌ GitHub Actions cache miss — started from a cold store." + fi +fi + +{ + echo "## ❄️ Nix cache" + echo + if [ -n "$cache_line" ]; then + echo "$cache_line" + echo + fi + echo "Store paths by source for this run:" + echo + echo "| Source | Paths |" + echo "| --- | ---: |" + echo "| ♻️ Restored from GitHub Actions cache | $github_count |" + echo "| ⬇️ Substituted from upstream caches | $upstream_count |" + echo "| 🔨 Built locally (no cache) | $built_count |" + echo + + if [ "$built_count" -eq 0 ]; then + echo "Everything was served from a cache — nothing had to be built. 🎉" + elif [ "$built_count" -lt 100 ]; then + echo "
" + echo "🔨 Built locally ($built_count)" + echo + while IFS= read -r p; do + echo "- \`$p\`" + done < <(sort "$workdir/built.txt") + echo + echo "
" + else + echo "> $built_count paths were built locally (too many to list individually)." + fi + echo +} >> "$summary_file" diff --git a/scripts/cache-stats-snapshot.sh b/scripts/cache-stats-snapshot.sh new file mode 100755 index 0000000..74303dc --- /dev/null +++ b/scripts/cache-stats-snapshot.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +set -euo pipefail +export LC_ALL=C + +# Records the set of valid Nix store paths at a point in time, so the post-run +# report can attribute each derivation to a source. Two snapshots are taken: +# pre - after Nix is installed but *before* the cache is restored. This is the +# baseline toolchain (fetched from upstream during install); it's +# treated as noise and excluded from the report. +# post - immediately *after* the cache is restored, so (post - pre) is exactly +# what the GitHub Actions cache provided. +# The report itself takes a third look at the store in the post step, after the +# build has run, to find what the build added on top of this. + +phase="${1:?usage: cache-stats-snapshot.sh }" + +state_dir="${RUNNER_TEMP:-/tmp}/nix-magic-setup" +mkdir -p "$state_dir" + +store_paths() { + # Exclude .drv files: they're instantiated locally during evaluation, never + # "built" or "substituted", so counting them would only add noise. We report + # on realised outputs and their dependencies. + nix --extra-experimental-features nix-command path-info --all 2>/dev/null \ + | awk '!/\.drv$/' | sort -u +} + +case "$phase" in + pre) + store_paths > "$state_dir/pre-paths.txt" + ;; + post) + store_paths > "$state_dir/post-paths.txt" + # Stash the cache restore result so the report can show it alongside the + # derivation-level breakdown. + { + printf 'CACHE_HIT_PRIMARY_KEY=%s\n' "${CACHE_HIT_PRIMARY_KEY:-}" + printf 'CACHE_HIT_FIRST_MATCH=%s\n' "${CACHE_HIT_FIRST_MATCH:-}" + printf 'CACHE_PRIMARY_KEY=%s\n' "${CACHE_PRIMARY_KEY:-}" + printf 'CACHE_RESTORED_KEY=%s\n' "${CACHE_RESTORED_KEY:-}" + } > "$state_dir/cache-meta.env" + ;; + *) + echo "cache-stats-snapshot.sh: unknown phase '$phase'" >&2 + exit 1 + ;; +esac From 7b93a41bd83a555c8148ceee3f754fa17d88932a Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 10:20:51 +0000 Subject: [PATCH 02/21] Forward a change-sentinel to comment-flake-lock-changelog (testing) Add a `change-sentinel` input that is passed through to comment-flake-lock-changelog's `build-filter`, so flake.lock changelog commits that don't affect the build output can be filtered out of the PR comment. Temporarily pins comment-flake-lock-changelog to its main branch (its `build-filter` feature is unreleased) so the option can be exercised end-to-end. Revert to a tagged release once it ships. See mdarocha/comment-flake-lock-changelog#301. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- README.md | 1 + action.yml | 11 ++++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 0bb3187..4f7a84a 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,7 @@ jobs: | `token` | Github authentication token to use | `${{ github.token }}` | | `free-up-all-storage` | Aggressively free up all possible disk space on the runner before installing Nix, using [wimpysworld/nothing-but-nix](https://github.com/wimpysworld/nothing-but-nix) | `false` | | `max-cached-store-size` | Maximum (uncompressed) Nix store size to keep in the cache, e.g. `8G`, `512M`, or a plain byte count. Set to an empty string to disable garbage collection and cache the whole store. See [Cache size](#cache-size). | `8G` | +| `change-sentinel` | Optional shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`. Prints a fingerprint (a "change sentinel", e.g. a derivation's `drvPath`) for the flake input being tested, so `flake.lock` changelog commits that don't affect your build output are hidden from the PR comment. | `""` | ### Freeing up storage diff --git a/action.yml b/action.yml index e128f15..a7c723e 100644 --- a/action.yml +++ b/action.yml @@ -16,6 +16,10 @@ inputs: description: "Maximum size of the Nix store to keep in the cache, e.g. '8G', '512M', or a plain byte count. Just before a new cache is saved, older store paths are garbage-collected until the (uncompressed) Nix store is at or below this size. The cache uploaded to GitHub is compressed and is typically 2-4x smaller than this value. Set to an empty string to disable garbage collection and cache the whole store." required: false default: "8G" + change-sentinel: + description: "Optional shell command forwarded to comment-flake-lock-changelog's `build-filter`. It should print a fingerprint (a 'change sentinel', e.g. a derivation's drvPath) for the flake input being tested, so changelog commits that don't affect your build output are hidden from the PR comment. See comment-flake-lock-changelog's README for the command contract and the CFLC_INPUT_* environment variables." + required: false + default: "" runs: using: composite steps: @@ -83,8 +87,13 @@ runs: shell: bash run: bash "${{ github.action_path }}/scripts/setup-direnv.sh" - - uses: mdarocha/comment-flake-lock-changelog@709edb53d1a4937e9792876a036671f3316f1186 # v1.0.2 + # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of + # a release, to exercise its unreleased `build-filter` feature via the + # change-sentinel input. See comment-flake-lock-changelog#301. Revert to a + # tagged release once that feature ships. + - uses: mdarocha/comment-flake-lock-changelog@25e8004e64b9af48e6d61ccf2bfe875cf7bf14b6 # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} + build-filter: ${{ inputs.change-sentinel }} token: ${{ inputs.token }} From 10b5480c5541d0388d09d41a2707975cb752ae2c Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 10:32:03 +0000 Subject: [PATCH 03/21] Shorten input descriptions; keep detail in the README Address review feedback: keep action.yml input descriptions to a single line and leave the longer explanation to the README, sync the config table rows to the same short wording, and restore the unchanged cache-nix-action feature bullet. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- README.md | 7 +++---- action.yml | 4 ++-- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 4f7a84a..ba78b33 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,7 @@ single drop-in action. ## Features - Installing Nix using [cachix/install-nix-action](https://github.com/cachix/install-nix-action) -- Caching Nix derivations using [nix-community/cache-nix-action](https://github.com/nix-community/cache-nix-action), - with a tunable cache size that keeps build outputs warm across runs +- Caching Nix derivations using [nix-community/cache-nix-action](https://github.com/nix-community/cache-nix-action) - Reporting per-derivation cache stats to the GitHub Actions job summary — how many store paths were restored from the GitHub cache, substituted from upstream caches, or built locally - Automagically setting up environments from `.envrc` using direnv @@ -52,8 +51,8 @@ jobs: |--------------------|-------------------------------------------------------------------------------------------------------|----------------------| | `token` | Github authentication token to use | `${{ github.token }}` | | `free-up-all-storage` | Aggressively free up all possible disk space on the runner before installing Nix, using [wimpysworld/nothing-but-nix](https://github.com/wimpysworld/nothing-but-nix) | `false` | -| `max-cached-store-size` | Maximum (uncompressed) Nix store size to keep in the cache, e.g. `8G`, `512M`, or a plain byte count. Set to an empty string to disable garbage collection and cache the whole store. See [Cache size](#cache-size). | `8G` | -| `change-sentinel` | Optional shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`. Prints a fingerprint (a "change sentinel", e.g. a derivation's `drvPath`) for the flake input being tested, so `flake.lock` changelog commits that don't affect your build output are hidden from the PR comment. | `""` | +| `max-cached-store-size` | Maximum uncompressed Nix store size to keep in the cache (e.g. `8G`, `512M`); an empty string disables garbage collection. See [Cache size](#cache-size). | `8G` | +| `change-sentinel` | Shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`, to hide `flake.lock` changelog commits that don't affect your build output. See its README. | `""` | ### Freeing up storage diff --git a/action.yml b/action.yml index a7c723e..e3fa11c 100644 --- a/action.yml +++ b/action.yml @@ -13,11 +13,11 @@ inputs: required: false default: "false" max-cached-store-size: - description: "Maximum size of the Nix store to keep in the cache, e.g. '8G', '512M', or a plain byte count. Just before a new cache is saved, older store paths are garbage-collected until the (uncompressed) Nix store is at or below this size. The cache uploaded to GitHub is compressed and is typically 2-4x smaller than this value. Set to an empty string to disable garbage collection and cache the whole store." + description: "Maximum uncompressed Nix store size to keep in the cache (e.g. 8G, 512M); an empty string disables garbage collection. See the Cache size section in the README." required: false default: "8G" change-sentinel: - description: "Optional shell command forwarded to comment-flake-lock-changelog's `build-filter`. It should print a fingerprint (a 'change sentinel', e.g. a derivation's drvPath) for the flake input being tested, so changelog commits that don't affect your build output are hidden from the PR comment. See comment-flake-lock-changelog's README for the command contract and the CFLC_INPUT_* environment variables." + description: "Shell command forwarded to comment-flake-lock-changelog's build-filter, to hide flake.lock changelog commits that don't affect your build output. See comment-flake-lock-changelog's README." required: false default: "" runs: From b860b8681130ac16ba4c1113fde182c4c1820b93 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 10:48:26 +0000 Subject: [PATCH 04/21] Report cache restore/save details and sizes in the stats summary Extend the job-summary stats (per review feedback) to cover the whole cache lifecycle: - which cache was restored (exact primary vs a prefix match) and its size; - the save outcome: a new cache uploaded (size + delta vs the restored cache), skipped due to an exact primary-key hit, or a warning when no new cache is found afterwards. Cache sizes come from the GitHub Actions Cache API (needs actions: read, already required). Because cache-nix-action saves in its own post step and exposes no size outputs, the reporter is split into two post steps ordered around it: a capture step (registered after the cache step, so its post runs before the GC/save) snapshots the built store for the per-derivation breakdown, and the report step (registered before the cache step, so its post runs after the save) renders the summary and queries the API. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- README.md | 23 ++-- action.yml | 28 +++-- scripts/cache-stats-capture.sh | 26 +++++ scripts/cache-stats-report.sh | 191 +++++++++++++++++++-------------- 4 files changed, 176 insertions(+), 92 deletions(-) create mode 100755 scripts/cache-stats-capture.sh diff --git a/README.md b/README.md index ba78b33..4d3c26a 100644 --- a/README.md +++ b/README.md @@ -106,20 +106,29 @@ A few things worth knowing when tuning this: ### Cache stats -After your build steps have run, the action writes a breakdown to the +After your build steps have run, the action writes a summary to the [job summary](https://github.blog/news-insights/product-news/supercharging-github-actions-with-job-summaries/) -showing where each Nix store path came from: +covering the whole cache lifecycle. -- **♻️ Restored from GitHub Actions cache** — paths the [cache](#cache-size) restored, so they didn't - need to be fetched or built. +**Restore and save.** Which cache was restored — the exact **primary** cache or a **different** +prefix-matched one — and its size, followed by what happened on save: a new cache uploaded (with its +size and the delta versus the restored cache), or the save skipped because of an exact primary-key hit, +or a warning if no new cache turned up afterwards. Sizes come from the GitHub Actions Cache API, so the +job needs `actions: read` (already required by the [cache](#cache-size)). + +**Per-derivation breakdown.** Where each Nix store path came from: + +- **♻️ Restored from GitHub Actions cache** — paths the cache restored, so they didn't need to be + fetched or built. - **⬇️ Substituted from upstream caches** — paths pulled from binary caches (`cache.nixos.org`, Cachix, and any `extra-substituters` from your `flake.nix`) during the build. - **🔨 Built locally** — paths that were built on the runner because no cache had them. When there are fewer than 100, they're listed individually so you can see exactly what wasn't cached. -The breakdown works by snapshotting the store before and after the cache is restored, and classifying -whatever the build adds using Nix's own `ultimate` flag (set on locally-built paths). It runs in the -job's post phase — after your build — which the action arranges via +The breakdown snapshots the store before and after the cache is restored, and classifies whatever the +build adds using Nix's own `ultimate` flag (set on locally-built paths). Reporting runs in the job's +post phase — after your build, and ordered around cache-nix-action's own save so it can observe the +outcome — which the action arranges via [pyTooling/Actions/with-post-step](https://github.com/pyTooling/Actions), since a composite action can't declare a post step of its own. No configuration is required; it reports automatically. diff --git a/action.yml b/action.yml index e3fa11c..843f43b 100644 --- a/action.yml +++ b/action.yml @@ -41,6 +41,20 @@ runs: shell: bash run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" pre + # Registered BEFORE the cache step so its post step runs AFTER + # cache-nix-action's save (post steps run in reverse registration order). + # That lets the report observe the save outcome and query the Actions Cache + # API for cache sizes. Composite actions can't declare a post step of their + # own, so this borrows one via with-post-step. + - name: Report cache stats after build + uses: pyTooling/Actions/with-post-step@fdcb8e6fb145f72c8e2c9c3ff6160dc62cc3e22c # v7.9.0 + env: + NMS_TOKEN: ${{ inputs.token }} + with: + key: NMS_REPORT + main: 'echo "nix-magic-setup: Nix cache stats will be reported in the post step, after the build."' + post: 'bash "${{ github.action_path }}/scripts/cache-stats-report.sh"' + - name: Setup Nix Cache id: nix-cache uses: nix-community/cache-nix-action@7df957e333c1e5da7721f60227dbba6d06080569 # v7.0.2 @@ -65,14 +79,16 @@ runs: CACHE_RESTORED_KEY: ${{ steps.nix-cache.outputs.restored-key }} run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" post - # Register a post-phase step that runs after the workflow's build steps, so - # it can report which store paths came from where. Composite actions can't - # declare a post step of their own, so this borrows one via with-post-step. - - name: Report cache stats after build + # Registered AFTER the cache step so its post step runs BEFORE + # cache-nix-action's garbage collection and save. That captures the full + # built store for the per-derivation breakdown, before any paths are + # collected away. + - name: Capture Nix store after build uses: pyTooling/Actions/with-post-step@fdcb8e6fb145f72c8e2c9c3ff6160dc62cc3e22c # v7.9.0 with: - main: 'echo "nix-magic-setup: Nix cache stats will be reported in the post step, after the build."' - post: 'bash "${{ github.action_path }}/scripts/cache-stats-report.sh"' + key: NMS_CAPTURE + main: 'echo "nix-magic-setup: will capture the Nix store after the build for cache stats."' + post: 'bash "${{ github.action_path }}/scripts/cache-stats-capture.sh"' - name: Check for .envrc id: check-envrc diff --git a/scripts/cache-stats-capture.sh b/scripts/cache-stats-capture.sh new file mode 100755 index 0000000..baba730 --- /dev/null +++ b/scripts/cache-stats-capture.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +export LC_ALL=C + +# Captures the Nix store contents after the build but BEFORE cache-nix-action's +# garbage-collection/save runs, recording each path's `ultimate` flag (true = +# built locally, false = substituted). This runs in the job's post phase, and is +# ordered to execute before cache-nix-action's own post step (see action.yml), +# so the derivation-level stats reflect the full built store rather than the +# post-GC one. Best-effort: never fail the job. + +state_dir="${RUNNER_TEMP:-/tmp}/nix-magic-setup" +mkdir -p "$state_dir" + +# `path-info --json` is an object on newer Nix and an array on older Nix; +# normalise both to `\t` lines, excluding .drv files. +if ! nix --extra-experimental-features nix-command path-info --all --json 2>/dev/null \ + | jq -r ' + (if type == "object" then to_entries | map(.value + {path: .key}) else . end) + | .[] + | select((.path | endswith(".drv")) | not) + | [.path, (if .ultimate then "built" else "sub" end)] | @tsv + ' \ + | sort -u > "$state_dir/now.tsv"; then + : > "$state_dir/now.tsv" +fi diff --git a/scripts/cache-stats-report.sh b/scripts/cache-stats-report.sh index 730149c..94ed7d4 100755 --- a/scripts/cache-stats-report.sh +++ b/scripts/cache-stats-report.sh @@ -2,111 +2,144 @@ set -euo pipefail export LC_ALL=C -# Renders a per-derivation cache breakdown to the GitHub Actions job summary: -# how many store paths were restored from the GitHub Actions cache, substituted -# from upstream binary caches, or built locally during this run. Built paths are -# listed individually when there are fewer than 100 of them. +# Renders the Nix cache summary to the GitHub Actions job summary. It reports: +# - which cache was restored (primary vs a prefix match) and its size; +# - what happened on save: a new cache uploaded (size + delta vs the restored +# cache) or skipped (exact primary-key hit) or apparently failed; +# - a per-derivation breakdown: how many store paths came from the GitHub +# cache, upstream binary caches, or were built locally (listed when < 100). # -# Runs in the job's post phase (via pyTooling/Actions/with-post-step), after the -# workflow's build steps, so it can compare the store against the snapshots -# taken during setup. Everything here is best-effort: a failure must never fail -# the job, so the caller ignores the exit status. +# It runs in the job's post phase, ordered (see action.yml) to execute AFTER +# cache-nix-action's own save step, so the save outcome can be observed via the +# GitHub Actions Cache API. The built store is captured separately, before that +# GC/save, in cache-stats-capture.sh. Everything is best-effort: a failure must +# never fail the job, so the caller ignores the exit status. state_dir="${RUNNER_TEMP:-/tmp}/nix-magic-setup" summary_file="${GITHUB_STEP_SUMMARY:-/dev/stdout}" pre_file="$state_dir/pre-paths.txt" post_file="$state_dir/post-paths.txt" +now_file="$state_dir/now.tsv" meta_file="$state_dir/cache-meta.env" -if [ ! -f "$post_file" ]; then - echo "nix-magic-setup: no store snapshot found, skipping cache stats." - exit 0 +hit_primary=""; hit_first_match=""; primary_key=""; restored_key="" +if [ -f "$meta_file" ]; then + # shellcheck source=/dev/null + . "$meta_file" + hit_primary="${CACHE_HIT_PRIMARY_KEY:-}" + hit_first_match="${CACHE_HIT_FIRST_MATCH:-}" + primary_key="${CACHE_PRIMARY_KEY:-}" + restored_key="${CACHE_RESTORED_KEY:-}" fi -[ -f "$pre_file" ] || : > "$pre_file" -workdir="$(mktemp -d)" -trap 'rm -rf "$workdir"' EXIT +# --- GitHub Actions Cache API helpers --------------------------------------- +# Returns the size in bytes of the cache with the given exact key (largest match +# across refs), or empty if unknown / unavailable. +cache_size_bytes() { + local key="$1" + [ -n "$key" ] && [ -n "${NMS_TOKEN:-}" ] && [ -n "${GITHUB_REPOSITORY:-}" ] || return 0 + local api="${GITHUB_API_URL:-https://api.github.com}" + curl -sfSL \ + -H "Accept: application/vnd.github+json" \ + -H "Authorization: Bearer ${NMS_TOKEN}" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "${api}/repos/${GITHUB_REPOSITORY}/actions/caches?key=${key}" 2>/dev/null \ + | jq -r '[.actions_caches[]?.size_in_bytes] | (max // empty)' 2>/dev/null || true +} -tab="$(printf '\t')" +human() { + local b="${1:-}" + [ -n "$b" ] || { printf 'unknown'; return; } + numfmt --to=iec-i --suffix=B --format='%.1f' "$b" 2>/dev/null || printf '%s B' "$b" +} -# Current store contents with each path's `ultimate` flag. Nix sets -# ultimate=true for paths built locally and false for paths pulled from a binary -# cache, which is exactly the built-vs-substituted distinction we need. -# `path-info --json` is an object on newer Nix and an array on older Nix; -# normalise both to `\t` lines. -if ! nix --extra-experimental-features nix-command path-info --all --json 2>/dev/null \ - | jq -r ' - (if type == "object" then to_entries | map(.value + {path: .key}) else . end) - | .[] - | select((.path | endswith(".drv")) | not) - | [.path, (if .ultimate then "built" else "sub" end)] | @tsv - ' \ - | sort -u > "$workdir/now.tsv"; then - echo "nix-magic-setup: could not query the Nix store, skipping cache stats." - exit 0 +# --- Restore / save section -------------------------------------------------- +restored_size="$(cache_size_bytes "$restored_key")" + +if [ "$hit_primary" = "true" ]; then + restored_line="✅ Restored the **primary** cache (\`${restored_key}\`), size **$(human "$restored_size")**." +elif [ "$hit_first_match" = "true" ]; then + restored_line="♻️ Restored a **different** cache via prefix match (\`${restored_key}\`), size **$(human "$restored_size")**." +else + restored_line="❌ No cache restored — cold store." fi -cut -f1 "$workdir/now.tsv" > "$workdir/now.txt" -# What the GitHub Actions cache provided and is still present: (post - pre) ∩ now -comm -23 "$post_file" "$pre_file" | comm -12 - "$workdir/now.txt" > "$workdir/from-github.txt" +api_available=false +[ -n "${NMS_TOKEN:-}" ] && [ -n "${GITHUB_REPOSITORY:-}" ] && api_available=true -# What the build added on top of the restored store: now - post -comm -23 "$workdir/now.txt" "$post_file" > "$workdir/new.txt" +if [ "$hit_primary" = "true" ]; then + save_line="⏭️ Save skipped — exact primary-key hit, the restored cache is already current." +elif [ "$api_available" != "true" ]; then + save_line="⬆️ A new cache will be saved for \`${primary_key}\` (size unavailable — no Cache API access)." +else + saved_size="$(cache_size_bytes "$primary_key")" + if [ -n "$saved_size" ]; then + delta="" + if [ -n "$restored_size" ]; then + d=$(( saved_size - restored_size )) + sign="+"; [ "$d" -lt 0 ] && sign="-" + delta=" (${sign}$(human "${d#-}") vs restored)" + fi + save_line="⬆️ Saved a new cache (\`${primary_key}\`), size **$(human "$saved_size")**${delta}." + else + save_line="⚠️ No new cache found for \`${primary_key}\` after save — it may have failed or been skipped." + fi +fi -# Split the new paths into built-locally vs substituted-from-upstream using the -# ultimate flag captured above. -join -t "$tab" "$workdir/new.txt" "$workdir/now.tsv" > "$workdir/new-flagged.tsv" -awk -F'\t' '$2 == "built" { print $1 }' "$workdir/new-flagged.tsv" > "$workdir/built.txt" -awk -F'\t' '$2 == "sub" { print $1 }' "$workdir/new-flagged.tsv" > "$workdir/upstream.txt" +# --- Per-derivation breakdown ------------------------------------------------ +deriv_section="" +if [ -s "$now_file" ] && [ -f "$post_file" ]; then + [ -f "$pre_file" ] || : > "$pre_file" + workdir="$(mktemp -d)" + trap 'rm -rf "$workdir"' EXIT + tab="$(printf '\t')" -github_count=$(wc -l < "$workdir/from-github.txt" | tr -d ' ') -built_count=$(wc -l < "$workdir/built.txt" | tr -d ' ') -upstream_count=$(wc -l < "$workdir/upstream.txt" | tr -d ' ') + cut -f1 "$now_file" > "$workdir/now.txt" + comm -23 "$post_file" "$pre_file" | comm -12 - "$workdir/now.txt" > "$workdir/from-github.txt" + comm -23 "$workdir/now.txt" "$post_file" > "$workdir/new.txt" + join -t "$tab" "$workdir/new.txt" "$now_file" > "$workdir/new-flagged.tsv" + awk -F'\t' '$2 == "built" { print $1 }' "$workdir/new-flagged.tsv" > "$workdir/built.txt" + awk -F'\t' '$2 == "sub" { print $1 }' "$workdir/new-flagged.tsv" > "$workdir/upstream.txt" -# Cache restore status line. -cache_line="" -if [ -f "$meta_file" ]; then - # shellcheck source=/dev/null - . "$meta_file" - if [ "${CACHE_HIT_PRIMARY_KEY:-}" = "true" ]; then - cache_line="✅ GitHub Actions cache hit on the exact key (\`${CACHE_RESTORED_KEY:-}\`)." - elif [ "${CACHE_HIT_FIRST_MATCH:-}" = "true" ]; then - cache_line="♻️ GitHub Actions cache restored from a prefix match (\`${CACHE_RESTORED_KEY:-}\`)." - else - cache_line="❌ GitHub Actions cache miss — started from a cold store." - fi + github_count=$(wc -l < "$workdir/from-github.txt" | tr -d ' ') + built_count=$(wc -l < "$workdir/built.txt" | tr -d ' ') + upstream_count=$(wc -l < "$workdir/upstream.txt" | tr -d ' ') + + deriv_section="$( + echo "Store paths by source for this run:" + echo + echo "| Source | Paths |" + echo "| --- | ---: |" + echo "| ♻️ Restored from GitHub Actions cache | $github_count |" + echo "| ⬇️ Substituted from upstream caches | $upstream_count |" + echo "| 🔨 Built locally (no cache) | $built_count |" + echo + if [ "$built_count" -eq 0 ]; then + echo "Everything was served from a cache — nothing had to be built. 🎉" + elif [ "$built_count" -lt 100 ]; then + echo "
" + echo "🔨 Built locally ($built_count)" + echo + while IFS= read -r p; do echo "- \`$p\`"; done < <(sort "$workdir/built.txt") + echo + echo "
" + else + echo "> $built_count paths were built locally (too many to list individually)." + fi + )" fi +# --- Emit -------------------------------------------------------------------- { echo "## ❄️ Nix cache" echo - if [ -n "$cache_line" ]; then - echo "$cache_line" - echo - fi - echo "Store paths by source for this run:" - echo - echo "| Source | Paths |" - echo "| --- | ---: |" - echo "| ♻️ Restored from GitHub Actions cache | $github_count |" - echo "| ⬇️ Substituted from upstream caches | $upstream_count |" - echo "| 🔨 Built locally (no cache) | $built_count |" + echo "$restored_line" echo - - if [ "$built_count" -eq 0 ]; then - echo "Everything was served from a cache — nothing had to be built. 🎉" - elif [ "$built_count" -lt 100 ]; then - echo "
" - echo "🔨 Built locally ($built_count)" + echo "$save_line" + if [ -n "$deriv_section" ]; then echo - while IFS= read -r p; do - echo "- \`$p\`" - done < <(sort "$workdir/built.txt") - echo - echo "
" - else - echo "> $built_count paths were built locally (too many to list individually)." + echo "$deriv_section" fi echo } >> "$summary_file" From 54b9485dd6572e693ace8cfc158232f4db1a5612 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 18:58:24 +0000 Subject: [PATCH 05/21] Fix cache stats: primary-key output is never set; race-safe save detection Real-world smoke testing on pondinfra#141 (nix-magic-setup#23 review) surfaced two bugs: 1. The save-outcome line always rendered an empty key ("No new cache found for `` after save"). Root cause, confirmed in cache-nix-action's dist source: in this combined restore+save usage, its `primary-key` output is never actually set - `setState` only persists it as internal cross-phase state (core.saveState), not a step output, unlike hit-primary-key/hit-first-match/restored-key which are set directly. Fixed by computing the key ourselves from the same expression already given to the `primary-key` input, shared via a YAML anchor so it can't drift. 2. A real run showed "restored a different cache via prefix match" (a primary-key miss at restore) immediately followed by a "may have failed or been skipped" warning - but the job log showed cache-nix-action legitimately skipping the save because a cache for that exact key already existed by save time (saved by a concurrent workflow run evaluating the same flake.lock/*.nix state). cache-nix-action decides whether to save via its own check at save time, independent of the restore-time hit, so a restore miss doesn't imply this run's save succeeded or even ran. Fixed by having cache-stats-capture.sh (which already runs immediately before cache-nix-action's own save step) also check whether a cache for the primary key exists at that point; the report now distinguishes a genuine new upload from a save skipped because the key was already taken, and only warns on a real failure (neither before nor after). Also fixes a review nit: the change-sentinel row's "See its README" is now an actual link. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- README.md | 10 ++-- action.yml | 11 ++++- scripts/cache-stats-capture.sh | 20 ++++++++ scripts/cache-stats-report.sh | 84 +++++++++++++++++++--------------- scripts/lib/cache-api.sh | 27 +++++++++++ 5 files changed, 109 insertions(+), 43 deletions(-) create mode 100755 scripts/lib/cache-api.sh diff --git a/README.md b/README.md index 4d3c26a..8a94501 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ jobs: | `token` | Github authentication token to use | `${{ github.token }}` | | `free-up-all-storage` | Aggressively free up all possible disk space on the runner before installing Nix, using [wimpysworld/nothing-but-nix](https://github.com/wimpysworld/nothing-but-nix) | `false` | | `max-cached-store-size` | Maximum uncompressed Nix store size to keep in the cache (e.g. `8G`, `512M`); an empty string disables garbage collection. See [Cache size](#cache-size). | `8G` | -| `change-sentinel` | Shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`, to hide `flake.lock` changelog commits that don't affect your build output. See its README. | `""` | +| `change-sentinel` | Shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`, to hide `flake.lock` changelog commits that don't affect your build output. See its [README](https://github.com/mdarocha/comment-flake-lock-changelog#build-filter). | `""` | ### Freeing up storage @@ -112,9 +112,11 @@ covering the whole cache lifecycle. **Restore and save.** Which cache was restored — the exact **primary** cache or a **different** prefix-matched one — and its size, followed by what happened on save: a new cache uploaded (with its -size and the delta versus the restored cache), or the save skipped because of an exact primary-key hit, -or a warning if no new cache turned up afterwards. Sizes come from the GitHub Actions Cache API, so the -job needs `actions: read` (already required by the [cache](#cache-size)). +size and the delta versus the restored cache), the save skipped (either an exact primary-key hit at +restore, or a cache for that key already existing by save time — which happens when a concurrent run, +e.g. another workflow triggered by the same push, races to save the same key first), or a warning if no +cache turns up afterwards. Sizes come from the GitHub Actions Cache API, so the job needs `actions: read` +(already required by the [cache](#cache-size)). **Per-derivation breakdown.** Where each Nix store path came from: diff --git a/action.yml b/action.yml index 843f43b..f0f61ff 100644 --- a/action.yml +++ b/action.yml @@ -61,7 +61,7 @@ runs: with: nix: true save: true - primary-key: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} + primary-key: &primary-key nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} restore-prefixes-first-match: nix-${{ runner.os }}- gc-max-store-size: ${{ inputs.max-cached-store-size }} purge: true @@ -75,7 +75,11 @@ runs: env: CACHE_HIT_PRIMARY_KEY: ${{ steps.nix-cache.outputs.hit-primary-key }} CACHE_HIT_FIRST_MATCH: ${{ steps.nix-cache.outputs.hit-first-match }} - CACHE_PRIMARY_KEY: ${{ steps.nix-cache.outputs.primary-key }} + # cache-nix-action doesn't expose `primary-key` as a real step output in + # this combined restore+save usage (it only persists it as internal + # cross-phase state), so it's always empty - use the same literal key + # expression given to its `primary-key` input above instead. + CACHE_PRIMARY_KEY: *primary-key CACHE_RESTORED_KEY: ${{ steps.nix-cache.outputs.restored-key }} run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" post @@ -85,6 +89,9 @@ runs: # collected away. - name: Capture Nix store after build uses: pyTooling/Actions/with-post-step@fdcb8e6fb145f72c8e2c9c3ff6160dc62cc3e22c # v7.9.0 + env: + NMS_TOKEN: ${{ inputs.token }} + CACHE_PRIMARY_KEY: *primary-key with: key: NMS_CAPTURE main: 'echo "nix-magic-setup: will capture the Nix store after the build for cache stats."' diff --git a/scripts/cache-stats-capture.sh b/scripts/cache-stats-capture.sh index baba730..2eefbc9 100755 --- a/scripts/cache-stats-capture.sh +++ b/scripts/cache-stats-capture.sh @@ -8,6 +8,18 @@ export LC_ALL=C # ordered to execute before cache-nix-action's own post step (see action.yml), # so the derivation-level stats reflect the full built store rather than the # post-GC one. Best-effort: never fail the job. +# +# It also checks, at this same point, whether a cache for CACHE_PRIMARY_KEY +# already exists - i.e. immediately before cache-nix-action's own save step +# runs. cache-nix-action performs the same check internally to decide whether +# to upload a new cache, but doesn't expose that decision as an output. Doing +# our own check right beforehand lets the report distinguish "this run's save +# genuinely uploaded a new cache" from "a cache for this key already existed +# by save time" (e.g. a concurrent workflow run with the same flake.lock/*.nix +# state raced to save it first). + +script_dir="$(dirname "${BASH_SOURCE[0]}")" +source "$script_dir/lib/cache-api.sh" state_dir="${RUNNER_TEMP:-/tmp}/nix-magic-setup" mkdir -p "$state_dir" @@ -24,3 +36,11 @@ if ! nix --extra-experimental-features nix-command path-info --all --json 2>/dev | sort -u > "$state_dir/now.tsv"; then : > "$state_dir/now.tsv" fi + +pre_save_size="$(nms_cache_size_bytes "${CACHE_PRIMARY_KEY:-}")" +existed=false +[ -n "$pre_save_size" ] && existed=true +{ + printf 'CACHE_PRIMARY_EXISTED_PRE_SAVE=%s\n' "$existed" + printf 'CACHE_PRIMARY_SIZE_PRE_SAVE=%s\n' "$pre_save_size" +} > "$state_dir/pre-save.env" diff --git a/scripts/cache-stats-report.sh b/scripts/cache-stats-report.sh index 94ed7d4..5f75de8 100755 --- a/scripts/cache-stats-report.sh +++ b/scripts/cache-stats-report.sh @@ -5,15 +5,24 @@ export LC_ALL=C # Renders the Nix cache summary to the GitHub Actions job summary. It reports: # - which cache was restored (primary vs a prefix match) and its size; # - what happened on save: a new cache uploaded (size + delta vs the restored -# cache) or skipped (exact primary-key hit) or apparently failed; +# cache), skipped because a cache for the key already existed (whether from +# an exact primary-key hit at restore, or a concurrent run that saved one +# while this job was still building), or a warning if the save appears to +# have failed; # - a per-derivation breakdown: how many store paths came from the GitHub # cache, upstream binary caches, or were built locally (listed when < 100). # # It runs in the job's post phase, ordered (see action.yml) to execute AFTER # cache-nix-action's own save step, so the save outcome can be observed via the -# GitHub Actions Cache API. The built store is captured separately, before that -# GC/save, in cache-stats-capture.sh. Everything is best-effort: a failure must -# never fail the job, so the caller ignores the exit status. +# GitHub Actions Cache API - cache-nix-action exposes neither cache sizes nor +# the save decision as outputs. The built store, and whether a cache for the +# primary key already existed right before cache-nix-action's own save ran, +# are both captured separately in cache-stats-capture.sh, which runs just +# before that save. Everything here is best-effort: a failure must never fail +# the job, so the caller ignores the exit status. + +script_dir="$(dirname "${BASH_SOURCE[0]}")" +source "$script_dir/lib/cache-api.sh" state_dir="${RUNNER_TEMP:-/tmp}/nix-magic-setup" summary_file="${GITHUB_STEP_SUMMARY:-/dev/stdout}" @@ -22,6 +31,7 @@ pre_file="$state_dir/pre-paths.txt" post_file="$state_dir/post-paths.txt" now_file="$state_dir/now.tsv" meta_file="$state_dir/cache-meta.env" +pre_save_file="$state_dir/pre-save.env" hit_primary=""; hit_first_match=""; primary_key=""; restored_key="" if [ -f "$meta_file" ]; then @@ -33,61 +43,61 @@ if [ -f "$meta_file" ]; then restored_key="${CACHE_RESTORED_KEY:-}" fi -# --- GitHub Actions Cache API helpers --------------------------------------- -# Returns the size in bytes of the cache with the given exact key (largest match -# across refs), or empty if unknown / unavailable. -cache_size_bytes() { - local key="$1" - [ -n "$key" ] && [ -n "${NMS_TOKEN:-}" ] && [ -n "${GITHUB_REPOSITORY:-}" ] || return 0 - local api="${GITHUB_API_URL:-https://api.github.com}" - curl -sfSL \ - -H "Accept: application/vnd.github+json" \ - -H "Authorization: Bearer ${NMS_TOKEN}" \ - -H "X-GitHub-Api-Version: 2022-11-28" \ - "${api}/repos/${GITHUB_REPOSITORY}/actions/caches?key=${key}" 2>/dev/null \ - | jq -r '[.actions_caches[]?.size_in_bytes] | (max // empty)' 2>/dev/null || true -} +primary_existed_pre_save="false"; primary_size_pre_save="" +if [ -f "$pre_save_file" ]; then + # shellcheck source=/dev/null + . "$pre_save_file" + primary_existed_pre_save="${CACHE_PRIMARY_EXISTED_PRE_SAVE:-false}" + primary_size_pre_save="${CACHE_PRIMARY_SIZE_PRE_SAVE:-}" +fi -human() { - local b="${1:-}" - [ -n "$b" ] || { printf 'unknown'; return; } - numfmt --to=iec-i --suffix=B --format='%.1f' "$b" 2>/dev/null || printf '%s B' "$b" -} +api_available=false +[ -n "${NMS_TOKEN:-}" ] && [ -n "${GITHUB_REPOSITORY:-}" ] && api_available=true -# --- Restore / save section -------------------------------------------------- -restored_size="$(cache_size_bytes "$restored_key")" +# --- Restore section --------------------------------------------------------- +restored_size="$(nms_cache_size_bytes "$restored_key")" if [ "$hit_primary" = "true" ]; then - restored_line="✅ Restored the **primary** cache (\`${restored_key}\`), size **$(human "$restored_size")**." + restored_line="✅ Restored the **primary** cache (\`${restored_key}\`), size **$(nms_human_size "$restored_size")**." elif [ "$hit_first_match" = "true" ]; then - restored_line="♻️ Restored a **different** cache via prefix match (\`${restored_key}\`), size **$(human "$restored_size")**." + restored_line="♻️ Restored a **different** cache via prefix match (\`${restored_key}\`), size **$(nms_human_size "$restored_size")**." else restored_line="❌ No cache restored — cold store." fi -api_available=false -[ -n "${NMS_TOKEN:-}" ] && [ -n "${GITHUB_REPOSITORY:-}" ] && api_available=true - +# --- Save section ------------------------------------------------------------- +# cache-nix-action decides whether to save based on its OWN check, at save +# time, for whether a cache with the primary key already exists - independent +# of whether restore hit the primary key earlier in the job. So "restore +# missed the primary key" does not imply "this run saved a new cache": a +# concurrent run (e.g. another workflow triggered by the same push, evaluating +# the same flake.lock/*.nix state) can race to save that exact key first. +# `primary_existed_pre_save` (checked immediately before cache-nix-action's own +# save step) captures that race; comparing it against the current state below +# tells us what actually happened, without needing an output cache-nix-action +# doesn't provide. if [ "$hit_primary" = "true" ]; then - save_line="⏭️ Save skipped — exact primary-key hit, the restored cache is already current." + save_line="⏭️ Save skipped — exact primary-key hit at restore, the cache was already current." elif [ "$api_available" != "true" ]; then - save_line="⬆️ A new cache will be saved for \`${primary_key}\` (size unavailable — no Cache API access)." + save_line="⬆️ A new cache may be saved for \`${primary_key}\` (outcome unavailable — no Cache API access)." +elif [ "$primary_existed_pre_save" = "true" ]; then + save_line="⏭️ Save skipped — a cache for \`${primary_key}\` already existed by save time (size **$(nms_human_size "$primary_size_pre_save")**), likely uploaded by a concurrent run." else - saved_size="$(cache_size_bytes "$primary_key")" + saved_size="$(nms_cache_size_bytes "$primary_key")" if [ -n "$saved_size" ]; then delta="" if [ -n "$restored_size" ]; then d=$(( saved_size - restored_size )) sign="+"; [ "$d" -lt 0 ] && sign="-" - delta=" (${sign}$(human "${d#-}") vs restored)" + delta=" (${sign}$(nms_human_size "${d#-}") vs restored)" fi - save_line="⬆️ Saved a new cache (\`${primary_key}\`), size **$(human "$saved_size")**${delta}." + save_line="⬆️ Saved a new cache (\`${primary_key}\`), size **$(nms_human_size "$saved_size")**${delta}." else - save_line="⚠️ No new cache found for \`${primary_key}\` after save — it may have failed or been skipped." + save_line="⚠️ No cache found for \`${primary_key}\` after save — it may have failed." fi fi -# --- Per-derivation breakdown ------------------------------------------------ +# --- Per-derivation breakdown -------------------------------------------------- deriv_section="" if [ -s "$now_file" ] && [ -f "$post_file" ]; then [ -f "$pre_file" ] || : > "$pre_file" diff --git a/scripts/lib/cache-api.sh b/scripts/lib/cache-api.sh new file mode 100755 index 0000000..b82a33b --- /dev/null +++ b/scripts/lib/cache-api.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Shared helpers for querying the GitHub Actions Cache API. cache-nix-action +# doesn't expose cache sizes (or the save outcome) as step outputs, so the +# cache stats scripts fall back to querying the API directly wherever they +# need that information. + +# Prints the largest size in bytes (across refs) of a cache with the given +# exact key, or nothing if unknown, not found, or there's no API access +# (missing token/repository). Never fails the caller. +nms_cache_size_bytes() { + local key="$1" + [ -n "$key" ] && [ -n "${NMS_TOKEN:-}" ] && [ -n "${GITHUB_REPOSITORY:-}" ] || return 0 + local api="${GITHUB_API_URL:-https://api.github.com}" + curl -sfSL \ + -H "Accept: application/vnd.github+json" \ + -H "Authorization: Bearer ${NMS_TOKEN}" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "${api}/repos/${GITHUB_REPOSITORY}/actions/caches?key=${key}" 2>/dev/null \ + | jq -r '[.actions_caches[]?.size_in_bytes] | (max // empty)' 2>/dev/null || true +} + +# Renders a byte count as a human-readable IEC size, or "unknown" if empty. +nms_human_size() { + local b="${1:-}" + [ -n "$b" ] || { printf 'unknown'; return; } + numfmt --to=iec-i --suffix=B --format='%.1f' "$b" 2>/dev/null || printf '%s B' "$b" +} From 179b1066792996ce3542ac736bd7e73c1f33886a Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 19:11:56 +0000 Subject: [PATCH 06/21] Bump comment-flake-lock-changelog testing pin to latest main Picks up the node24 runtime fix (mdarocha/comment-flake-lock-changelog#315, merged) so the temporary main pin used for testing change-sentinel no longer triggers the Node.js 20 deprecation warning. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- action.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/action.yml b/action.yml index f0f61ff..6e81080 100644 --- a/action.yml +++ b/action.yml @@ -114,7 +114,7 @@ runs: # a release, to exercise its unreleased `build-filter` feature via the # change-sentinel input. See comment-flake-lock-changelog#301. Revert to a # tagged release once that feature ships. - - uses: mdarocha/comment-flake-lock-changelog@25e8004e64b9af48e6d61ccf2bfe875cf7bf14b6 # main + - uses: mdarocha/comment-flake-lock-changelog@cc4034438e22d4dfc90741d6df0a96a3bc6dd1e9 # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From f14bc8d9a3f58eb92491d33a2c631e5b11e12cb6 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 19:14:36 +0000 Subject: [PATCH 07/21] Fix action load failure: GitHub's action.yml parser rejects YAML anchors The previous commit's YAML anchor/alias (to share the cache primary-key expression across three steps without duplicating it) broke the action entirely: ##[error]mdarocha/nix-magic-setup/.../action.yml: Anchors are not currently supported. Remove the anchor 'primary-key' Confirmed via a real smoke-test run on pondinfra#141 - the job failed at "Set up job", before any step ran. Local YAML validation (PyYAML) didn't catch this because it resolves anchors just fine; GitHub's own action manifest parser is stricter than generic YAML. Reverted to spelling out the literal key expression in all three places, with a comment explaining why and where to keep them in sync. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- action.yml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/action.yml b/action.yml index 6e81080..00761b1 100644 --- a/action.yml +++ b/action.yml @@ -61,7 +61,7 @@ runs: with: nix: true save: true - primary-key: &primary-key nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} + primary-key: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} restore-prefixes-first-match: nix-${{ runner.os }}- gc-max-store-size: ${{ inputs.max-cached-store-size }} purge: true @@ -78,8 +78,12 @@ runs: # cache-nix-action doesn't expose `primary-key` as a real step output in # this combined restore+save usage (it only persists it as internal # cross-phase state), so it's always empty - use the same literal key - # expression given to its `primary-key` input above instead. - CACHE_PRIMARY_KEY: *primary-key + # expression given to its `primary-key` input above instead. (A YAML + # anchor/alias would avoid this duplication, but GitHub's action.yml + # parser rejects anchors outright, so it has to be spelled out here + # and in the "Capture Nix store after build" step below - keep both + # in sync with the `primary-key:` input above if this ever changes.) + CACHE_PRIMARY_KEY: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} CACHE_RESTORED_KEY: ${{ steps.nix-cache.outputs.restored-key }} run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" post @@ -91,7 +95,7 @@ runs: uses: pyTooling/Actions/with-post-step@fdcb8e6fb145f72c8e2c9c3ff6160dc62cc3e22c # v7.9.0 env: NMS_TOKEN: ${{ inputs.token }} - CACHE_PRIMARY_KEY: *primary-key + CACHE_PRIMARY_KEY: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} with: key: NMS_CAPTURE main: 'echo "nix-magic-setup: will capture the Nix store after the build for cache stats."' From 432444e785041a58ef20245277c010e4a5550103 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Thu, 23 Jul 2026 19:35:19 +0000 Subject: [PATCH 08/21] Rename change-sentinel input to changelog-filter Per review feedback - it forwards to comment-flake-lock-changelog's build-filter, and "changelog-filter" names what it does more directly. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_016MYz49D9GAahQN6haYW3Nb --- README.md | 2 +- action.yml | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8a94501..2fc4f34 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ jobs: | `token` | Github authentication token to use | `${{ github.token }}` | | `free-up-all-storage` | Aggressively free up all possible disk space on the runner before installing Nix, using [wimpysworld/nothing-but-nix](https://github.com/wimpysworld/nothing-but-nix) | `false` | | `max-cached-store-size` | Maximum uncompressed Nix store size to keep in the cache (e.g. `8G`, `512M`); an empty string disables garbage collection. See [Cache size](#cache-size). | `8G` | -| `change-sentinel` | Shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`, to hide `flake.lock` changelog commits that don't affect your build output. See its [README](https://github.com/mdarocha/comment-flake-lock-changelog#build-filter). | `""` | +| `changelog-filter` | Shell command forwarded to [comment-flake-lock-changelog](https://github.com/mdarocha/comment-flake-lock-changelog)'s `build-filter`, to hide `flake.lock` changelog commits that don't affect your build output. See its [README](https://github.com/mdarocha/comment-flake-lock-changelog#build-filter). | `""` | ### Freeing up storage diff --git a/action.yml b/action.yml index 00761b1..c0a8f4b 100644 --- a/action.yml +++ b/action.yml @@ -16,7 +16,7 @@ inputs: description: "Maximum uncompressed Nix store size to keep in the cache (e.g. 8G, 512M); an empty string disables garbage collection. See the Cache size section in the README." required: false default: "8G" - change-sentinel: + changelog-filter: description: "Shell command forwarded to comment-flake-lock-changelog's build-filter, to hide flake.lock changelog commits that don't affect your build output. See comment-flake-lock-changelog's README." required: false default: "" @@ -116,11 +116,11 @@ runs: # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of # a release, to exercise its unreleased `build-filter` feature via the - # change-sentinel input. See comment-flake-lock-changelog#301. Revert to a + # changelog-filter input. See comment-flake-lock-changelog#301. Revert to a # tagged release once that feature ships. - uses: mdarocha/comment-flake-lock-changelog@cc4034438e22d4dfc90741d6df0a96a3bc6dd1e9 # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} - build-filter: ${{ inputs.change-sentinel }} + build-filter: ${{ inputs.changelog-filter }} token: ${{ inputs.token }} From 26964fb8346d83983510ae2f7a37c23d3d785c46 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 07:56:37 +0000 Subject: [PATCH 09/21] Bump comment-flake-lock-changelog testing pin to bd80de7 Picks up #317 (changelog comment formatting) and #318 (build-filter now bisects against the actual head commit and paginates past the compare API's 250-commit cap, instead of silently bisecting only the first 250 commits of a range). --- action.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/action.yml b/action.yml index c0a8f4b..4ed1e0a 100644 --- a/action.yml +++ b/action.yml @@ -118,7 +118,7 @@ runs: # a release, to exercise its unreleased `build-filter` feature via the # changelog-filter input. See comment-flake-lock-changelog#301. Revert to a # tagged release once that feature ships. - - uses: mdarocha/comment-flake-lock-changelog@cc4034438e22d4dfc90741d6df0a96a3bc6dd1e9 # main + - uses: mdarocha/comment-flake-lock-changelog@bd80de79e79c55ce5f6d2d335d769ddb1bc52638 # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From bdbcc0246a57733c1297ade621c9436e0c125a3c Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 09:10:34 +0000 Subject: [PATCH 10/21] Bump comment-flake-lock-changelog testing pin to 5c7ef97 Picks up #321: build-filter now resolves CFLC_INPUT_NAME to the flake's real input path (instead of the flake.lock node key, which silently no-ops --override-input for deduplicated inputs like nixpkgs), and no longer crashes with EPIPE on large commit ranges. --- action.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/action.yml b/action.yml index 4ed1e0a..1e1db74 100644 --- a/action.yml +++ b/action.yml @@ -118,7 +118,7 @@ runs: # a release, to exercise its unreleased `build-filter` feature via the # changelog-filter input. See comment-flake-lock-changelog#301. Revert to a # tagged release once that feature ships. - - uses: mdarocha/comment-flake-lock-changelog@bd80de79e79c55ce5f6d2d335d769ddb1bc52638 # main + - uses: mdarocha/comment-flake-lock-changelog@5c7ef9791f703d4eb174529ba6977e3a34c888dd # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From 05db4f92ff8653386ee52733a94a09d2c065eb85 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 12:45:49 +0000 Subject: [PATCH 11/21] Run changelog-filter before the cache restore, with build-filter-gc enabled The nix-magic-setup smoke test on a real nixpkgs bump (2500+ commits) hit "no space left on device" in a later, unrelated step once build-filter's classification bug was fixed and it started doing real work: bisecting a large range with comment-flake-lock-changelog's recommended path: fetcher usage copies the entire checked-out input into the Nix store on every build, and nothing reclaimed those copies before the rest of the job needed the disk. comment-flake-lock-changelog#322 adds an opt-in build-filter-gc input that runs `nix store gc` between builds to bound that growth, but it's only safe to use if nothing else in the job depends on Nix store paths that aren't rooted yet - a cache that was merely *restored* isn't necessarily a GC root, so gc'ing after that restore risks deleting the cache that was just pulled in. Moving this step to run before the cache is restored (instead of near the end, after the cache/build steps) makes build-filter-gc: true safe: the store is still essentially empty at this point, so there's nothing valuable for the between-build gc to collect away. --- README.md | 10 ++++++++++ action.yml | 29 ++++++++++++++++++----------- 2 files changed, 28 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 2fc4f34..f8fbf5d 100644 --- a/README.md +++ b/README.md @@ -134,6 +134,16 @@ outcome — which the action arranges via [pyTooling/Actions/with-post-step](https://github.com/pyTooling/Actions), since a composite action can't declare a post step of its own. No configuration is required; it reports automatically. +### Changelog filter + +`changelog-filter` runs before the Nix store cache is restored, not after — deliberately. Determining +build relevance checks out the changed input (e.g. `nixpkgs`) at various commits, and each check +gets copied into the Nix store fresh, uncached, so a wide-reaching range (a multi-day `nixpkgs` bump +can be thousands of commits) can use a meaningful amount of disk before it's done. It also runs with +comment-flake-lock-changelog's `build-filter-gc: true`, so it garbage-collects between each of those +checks — safe to do here specifically because the Nix store is still essentially empty at this point +(nothing has been restored from the cache yet for it to accidentally collect away). + ## Permissions required This action uses the workflows' `GITHUB_TOKEN` by default. Certain features require specific permissions to work. diff --git a/action.yml b/action.yml index 1e1db74..38149be 100644 --- a/action.yml +++ b/action.yml @@ -37,6 +37,24 @@ runs: shell: bash run: bash "${{ github.action_path }}/scripts/setup-nixconfig.sh" + # Runs here - before the cache is restored - rather than near the end of the + # steps below, so its build-filter-gc: true is actually safe to use: the store + # is still essentially empty at this point (just the Nix installation itself), + # so nix store gc between build-filter's builds can't collect away anything the + # rest of this job needs. build-filter's own README warns that running this + # after a cache restore risks gc deleting the very cache that was just restored. + # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of + # a release, to exercise its unreleased `build-filter`/`build-filter-gc` + # features via the changelog-filter input. See comment-flake-lock-changelog#301. + # Revert to a tagged release once those features ship. + - uses: mdarocha/comment-flake-lock-changelog@5c7ef9791f703d4eb174529ba6977e3a34c888dd # main + if: github.event_name == 'pull_request' + with: + pull-request-number: ${{ github.event.pull_request.number }} + build-filter: ${{ inputs.changelog-filter }} + build-filter-gc: "true" + token: ${{ inputs.token }} + - name: Snapshot Nix store before cache restore shell: bash run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" pre @@ -113,14 +131,3 @@ runs: if: steps.check-envrc.outputs.exists == 'true' shell: bash run: bash "${{ github.action_path }}/scripts/setup-direnv.sh" - - # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of - # a release, to exercise its unreleased `build-filter` feature via the - # changelog-filter input. See comment-flake-lock-changelog#301. Revert to a - # tagged release once that feature ships. - - uses: mdarocha/comment-flake-lock-changelog@5c7ef9791f703d4eb174529ba6977e3a34c888dd # main - if: github.event_name == 'pull_request' - with: - pull-request-number: ${{ github.event.pull_request.number }} - build-filter: ${{ inputs.changelog-filter }} - token: ${{ inputs.token }} From c619005339ca3cbdad7b413c8c852841411542a5 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 12:46:31 +0000 Subject: [PATCH 12/21] Pin comment-flake-lock-changelog to claude/build-filter-gc for testing Points at #322's branch (build-filter-gc: true isn't in a merged commit yet) so this PR's smoke test actually exercises it instead of silently no-op'ing against a pin that predates the feature. --- action.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/action.yml b/action.yml index 38149be..1687e23 100644 --- a/action.yml +++ b/action.yml @@ -43,11 +43,12 @@ runs: # so nix store gc between build-filter's builds can't collect away anything the # rest of this job needs. build-filter's own README warns that running this # after a cache restore risks gc deleting the very cache that was just restored. - # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of + # TEMPORARY (testing): pinned to comment-flake-lock-changelog's + # claude/build-filter-gc branch (comment-flake-lock-changelog#322) instead of # a release, to exercise its unreleased `build-filter`/`build-filter-gc` # features via the changelog-filter input. See comment-flake-lock-changelog#301. # Revert to a tagged release once those features ship. - - uses: mdarocha/comment-flake-lock-changelog@5c7ef9791f703d4eb174529ba6977e3a34c888dd # main + - uses: mdarocha/comment-flake-lock-changelog@ee59182e2551fdc7dfa302f85a0a5d6551b0ef35 # claude/build-filter-gc if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From c7a0252125df0c3e993531d3e468e250ec6143fc Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 12:54:18 +0000 Subject: [PATCH 13/21] Use build-filter-skip-checkout + git+file://?rev= to halve disk usage Pairs with comment-flake-lock-changelog#322: build-filter-gc alone still reclaims one full "checkout + store copy" per commit before moving to the next; build-filter-skip-checkout avoids the checkout copy entirely by having Nix read each commit straight out of the local clone's git object database via git+file://?rev= instead of path:, which needs this action to check the commit out to disk first. Updated here since deploy.yml's changelog-filter command needs to switch to that pattern for skip-checkout to be safe to enable. --- README.md | 15 +++++++++------ action.yml | 17 +++++++++++++---- 2 files changed, 22 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index f8fbf5d..fd8caf7 100644 --- a/README.md +++ b/README.md @@ -137,12 +137,15 @@ can't declare a post step of its own. No configuration is required; it reports a ### Changelog filter `changelog-filter` runs before the Nix store cache is restored, not after — deliberately. Determining -build relevance checks out the changed input (e.g. `nixpkgs`) at various commits, and each check -gets copied into the Nix store fresh, uncached, so a wide-reaching range (a multi-day `nixpkgs` bump -can be thousands of commits) can use a meaningful amount of disk before it's done. It also runs with -comment-flake-lock-changelog's `build-filter-gc: true`, so it garbage-collects between each of those -checks — safe to do here specifically because the Nix store is still essentially empty at this point -(nothing has been restored from the cache yet for it to accidentally collect away). +build relevance evaluates the changed input (e.g. `nixpkgs`) at various commits, and each one gets +imported into the Nix store fresh, uncached, so a wide-reaching range (a multi-day `nixpkgs` bump can +be thousands of commits) can use a meaningful amount of disk before it's done. It also runs with two +of comment-flake-lock-changelog's disk-space options: `build-filter-skip-checkout: true`, so each +commit goes straight from the local git clone's object database into the Nix store instead of being +checked out to disk first and then copied a second time, and `build-filter-gc: true`, which garbage- +collects between builds on top of that. Both are only safe to run here, before the cache is restored +— running them after would risk collecting away the cache that was just restored, since a merely- +*restored* store path isn't necessarily a GC root. ## Permissions required diff --git a/action.yml b/action.yml index 1687e23..41552cf 100644 --- a/action.yml +++ b/action.yml @@ -45,15 +45,24 @@ runs: # after a cache restore risks gc deleting the very cache that was just restored. # TEMPORARY (testing): pinned to comment-flake-lock-changelog's # claude/build-filter-gc branch (comment-flake-lock-changelog#322) instead of - # a release, to exercise its unreleased `build-filter`/`build-filter-gc` - # features via the changelog-filter input. See comment-flake-lock-changelog#301. - # Revert to a tagged release once those features ship. - - uses: mdarocha/comment-flake-lock-changelog@ee59182e2551fdc7dfa302f85a0a5d6551b0ef35 # claude/build-filter-gc + # a release, to exercise its unreleased `build-filter`/`build-filter-gc`/ + # `build-filter-skip-checkout` features via the changelog-filter input. See + # comment-flake-lock-changelog#301. Revert to a tagged release once those + # features ship. + # + # build-filter-skip-checkout: true pairs with the changelog-filter command + # itself using git+file://$CFLC_INPUT_PATH?rev=$CFLC_INPUT_REV rather than + # path:$CFLC_INPUT_PATH, so Nix reads each commit straight out of the repo's + # object database instead of this action checking it out to disk first just to + # have Nix copy that checkout into the store again - see + # comment-flake-lock-changelog's README's 'Disk space' section. + - uses: mdarocha/comment-flake-lock-changelog@1e4da2277c2f64ee03f4df6f78713e13cebff441 # claude/build-filter-gc if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} build-filter: ${{ inputs.changelog-filter }} build-filter-gc: "true" + build-filter-skip-checkout: "true" token: ${{ inputs.token }} - name: Snapshot Nix store before cache restore From f12cb14cf0d51e28f9efbb8da39c2fc7205a5047 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 13:07:15 +0000 Subject: [PATCH 14/21] Drop build-filter-skip-checkout: it doesn't work comment-flake-lock-changelog reverted it (comment-flake-lock-changelog#322) after a real run on pondinfra#141 failed with a libgit2 "object not found" error: Nix's git fetcher can't lazily fetch missing blobs from the blobless clone's promisor remote the way `git checkout` (the real git CLI) can, so skipping that checkout leaves libgit2 unable to find what it needs. Keeps build-filter-gc and the git+file://?rev= pattern in changelog-filter, both of which are unaffected - the checkout still runs, it just no longer gets skipped. --- README.md | 12 +++++------- action.yml | 25 ++++++++++++++----------- 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index fd8caf7..3e6a8d2 100644 --- a/README.md +++ b/README.md @@ -139,13 +139,11 @@ can't declare a post step of its own. No configuration is required; it reports a `changelog-filter` runs before the Nix store cache is restored, not after — deliberately. Determining build relevance evaluates the changed input (e.g. `nixpkgs`) at various commits, and each one gets imported into the Nix store fresh, uncached, so a wide-reaching range (a multi-day `nixpkgs` bump can -be thousands of commits) can use a meaningful amount of disk before it's done. It also runs with two -of comment-flake-lock-changelog's disk-space options: `build-filter-skip-checkout: true`, so each -commit goes straight from the local git clone's object database into the Nix store instead of being -checked out to disk first and then copied a second time, and `build-filter-gc: true`, which garbage- -collects between builds on top of that. Both are only safe to run here, before the cache is restored -— running them after would risk collecting away the cache that was just restored, since a merely- -*restored* store path isn't necessarily a GC root. +be thousands of commits) can use a meaningful amount of disk before it's done. It also runs with +comment-flake-lock-changelog's `build-filter-gc: true`, so it garbage-collects between each of those +builds — only safe to run here, before the cache is restored, since running it after would risk +collecting away the cache that was just restored (a merely-*restored* store path isn't necessarily a +GC root). ## Permissions required diff --git a/action.yml b/action.yml index 41552cf..c95b1bd 100644 --- a/action.yml +++ b/action.yml @@ -45,24 +45,27 @@ runs: # after a cache restore risks gc deleting the very cache that was just restored. # TEMPORARY (testing): pinned to comment-flake-lock-changelog's # claude/build-filter-gc branch (comment-flake-lock-changelog#322) instead of - # a release, to exercise its unreleased `build-filter`/`build-filter-gc`/ - # `build-filter-skip-checkout` features via the changelog-filter input. See - # comment-flake-lock-changelog#301. Revert to a tagged release once those - # features ship. + # a release, to exercise its unreleased `build-filter`/`build-filter-gc` + # features via the changelog-filter input. See comment-flake-lock-changelog#301. + # Revert to a tagged release once those features ship. # - # build-filter-skip-checkout: true pairs with the changelog-filter command - # itself using git+file://$CFLC_INPUT_PATH?rev=$CFLC_INPUT_REV rather than - # path:$CFLC_INPUT_PATH, so Nix reads each commit straight out of the repo's - # object database instead of this action checking it out to disk first just to - # have Nix copy that checkout into the store again - see + # NOTE: build-filter-skip-checkout was tried here too, paired with the + # changelog-filter command using git+file://$CFLC_INPUT_PATH?rev=$CFLC_INPUT_REV, + # but it doesn't work: Nix's git fetcher is built on libgit2, which can't + # lazily fetch missing blobs from the blobless clone's promisor remote the way + # `git checkout` (the real git CLI) can, so skipping that checkout makes Nix + # fail with "object not found" the moment it needs a blob nothing ever fetched. + # The changelog-filter command still uses git+file://...?rev=... below, just + # without skip-checkout - the checkout still runs and fetches what libgit2 + # needs, and Nix reads it from the object database instead of re-copying the + # checked-out working tree the way path:$CFLC_INPUT_PATH would. See # comment-flake-lock-changelog's README's 'Disk space' section. - - uses: mdarocha/comment-flake-lock-changelog@1e4da2277c2f64ee03f4df6f78713e13cebff441 # claude/build-filter-gc + - uses: mdarocha/comment-flake-lock-changelog@01f6875da8ffd324c9e7e135f1faa57555c77744 # claude/build-filter-gc if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} build-filter: ${{ inputs.changelog-filter }} build-filter-gc: "true" - build-filter-skip-checkout: "true" token: ${{ inputs.token }} - name: Snapshot Nix store before cache restore From 17e658ee8bbe8810b020dc88a1fee982925e0960 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 13:47:57 +0000 Subject: [PATCH 15/21] Bump comment-flake-lock-changelog testing pin to 5d4a1c6 Picks up #322 (build-filter-gc) and #323 (fixes the GitHub API result cache, which never actually persisted across runs due to an immutable, unversioned cache key), both merged to main. --- action.yml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/action.yml b/action.yml index c95b1bd..d232ab7 100644 --- a/action.yml +++ b/action.yml @@ -43,8 +43,7 @@ runs: # so nix store gc between build-filter's builds can't collect away anything the # rest of this job needs. build-filter's own README warns that running this # after a cache restore risks gc deleting the very cache that was just restored. - # TEMPORARY (testing): pinned to comment-flake-lock-changelog's - # claude/build-filter-gc branch (comment-flake-lock-changelog#322) instead of + # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of # a release, to exercise its unreleased `build-filter`/`build-filter-gc` # features via the changelog-filter input. See comment-flake-lock-changelog#301. # Revert to a tagged release once those features ship. @@ -60,7 +59,7 @@ runs: # needs, and Nix reads it from the object database instead of re-copying the # checked-out working tree the way path:$CFLC_INPUT_PATH would. See # comment-flake-lock-changelog's README's 'Disk space' section. - - uses: mdarocha/comment-flake-lock-changelog@01f6875da8ffd324c9e7e135f1faa57555c77744 # claude/build-filter-gc + - uses: mdarocha/comment-flake-lock-changelog@5d4a1c6c26712284a7632c9f944bf24d9e70981e # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From e6e7e960f1b2b5cfb3ddc60a54d224553831e9b6 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 14:52:07 +0000 Subject: [PATCH 16/21] Bump uncached-build listing limit to 200, bust cache on gc size change Raises the per-derivation summary's "list individually" threshold for locally-built paths from 100 to 200, matching what the summary reports above it. Also folds max-cached-store-size into the Nix store cache's primary key. Previously, changing that config value alone (nix/flake.lock unchanged) still hit the same primary key, so cache-nix-action would report an exact-match "nothing to do" even though the target gc size had changed. Keying on it too means a config change always saves a fresh cache under the new target. --- README.md | 5 ++++- action.yml | 6 +++--- scripts/cache-stats-report.sh | 4 ++-- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 3e6a8d2..14fc9cc 100644 --- a/README.md +++ b/README.md @@ -97,6 +97,9 @@ A few things worth knowing when tuning this: - **Disabling collection.** Set `max-cached-store-size` to an empty string to skip garbage collection entirely and cache the whole store. Only do this if you're confident the store stays comfortably under GitHub's cache limits. +- **Changing this value busts the cache.** It's part of the cache's primary key, so a run with a new + `max-cached-store-size` always saves a fresh cache under the new target instead of reusing one + collected under the old one. ```yaml - uses: mdarocha/nix-magic-setup@v1.1.0 @@ -125,7 +128,7 @@ cache turns up afterwards. Sizes come from the GitHub Actions Cache API, so the - **⬇️ Substituted from upstream caches** — paths pulled from binary caches (`cache.nixos.org`, Cachix, and any `extra-substituters` from your `flake.nix`) during the build. - **🔨 Built locally** — paths that were built on the runner because no cache had them. When there are - fewer than 100, they're listed individually so you can see exactly what wasn't cached. + fewer than 200, they're listed individually so you can see exactly what wasn't cached. The breakdown snapshots the store before and after the cache is restored, and classifies whatever the build adds using Nix's own `ultimate` flag (set on locally-built paths). Reporting runs in the job's diff --git a/action.yml b/action.yml index d232ab7..50f5b58 100644 --- a/action.yml +++ b/action.yml @@ -91,7 +91,7 @@ runs: with: nix: true save: true - primary-key: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} + primary-key: nix-${{ runner.os }}-${{ inputs.max-cached-store-size }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} restore-prefixes-first-match: nix-${{ runner.os }}- gc-max-store-size: ${{ inputs.max-cached-store-size }} purge: true @@ -113,7 +113,7 @@ runs: # parser rejects anchors outright, so it has to be spelled out here # and in the "Capture Nix store after build" step below - keep both # in sync with the `primary-key:` input above if this ever changes.) - CACHE_PRIMARY_KEY: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} + CACHE_PRIMARY_KEY: nix-${{ runner.os }}-${{ inputs.max-cached-store-size }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} CACHE_RESTORED_KEY: ${{ steps.nix-cache.outputs.restored-key }} run: bash "${{ github.action_path }}/scripts/cache-stats-snapshot.sh" post @@ -125,7 +125,7 @@ runs: uses: pyTooling/Actions/with-post-step@fdcb8e6fb145f72c8e2c9c3ff6160dc62cc3e22c # v7.9.0 env: NMS_TOKEN: ${{ inputs.token }} - CACHE_PRIMARY_KEY: nix-${{ runner.os }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} + CACHE_PRIMARY_KEY: nix-${{ runner.os }}-${{ inputs.max-cached-store-size }}-${{ hashFiles('**/*.nix', '**/flake.lock') }} with: key: NMS_CAPTURE main: 'echo "nix-magic-setup: will capture the Nix store after the build for cache stats."' diff --git a/scripts/cache-stats-report.sh b/scripts/cache-stats-report.sh index 5f75de8..0dcc421 100755 --- a/scripts/cache-stats-report.sh +++ b/scripts/cache-stats-report.sh @@ -10,7 +10,7 @@ export LC_ALL=C # while this job was still building), or a warning if the save appears to # have failed; # - a per-derivation breakdown: how many store paths came from the GitHub -# cache, upstream binary caches, or were built locally (listed when < 100). +# cache, upstream binary caches, or were built locally (listed when < 200). # # It runs in the job's post phase, ordered (see action.yml) to execute AFTER # cache-nix-action's own save step, so the save outcome can be observed via the @@ -127,7 +127,7 @@ if [ -s "$now_file" ] && [ -f "$post_file" ]; then echo if [ "$built_count" -eq 0 ]; then echo "Everything was served from a cache — nothing had to be built. 🎉" - elif [ "$built_count" -lt 100 ]; then + elif [ "$built_count" -lt 200 ]; then echo "
" echo "🔨 Built locally ($built_count)" echo From 30b8cbb256ba115b1cb877b25362500133c70554 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 18:40:36 +0000 Subject: [PATCH 17/21] Bump comment-flake-lock-changelog testing pin to 5b367a2 Picks up the EPIPE fix for the per-commit PR-lookup logging loop, found live on pondinfra#141 with a 1847-commit range. --- action.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/action.yml b/action.yml index 50f5b58..7637b72 100644 --- a/action.yml +++ b/action.yml @@ -59,7 +59,7 @@ runs: # needs, and Nix reads it from the object database instead of re-copying the # checked-out working tree the way path:$CFLC_INPUT_PATH would. See # comment-flake-lock-changelog's README's 'Disk space' section. - - uses: mdarocha/comment-flake-lock-changelog@5d4a1c6c26712284a7632c9f944bf24d9e70981e # main + - uses: mdarocha/comment-flake-lock-changelog@5b367a28257ccae64ac9e4bacca51e2549534a14 # claude/build-filter-result-cache if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From 9de4a73faaecfe5a635d213cd7b615db14a1d848 Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 18:50:19 +0000 Subject: [PATCH 18/21] Bump comment-flake-lock-changelog testing pin to 8b4c27f Picks up the fix for eager bulk-fetching the whole commit range instead of each commit lazily per build, found live on pondinfra#141 with a 1847-commit nixpkgs bump. --- action.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/action.yml b/action.yml index 7637b72..56d873c 100644 --- a/action.yml +++ b/action.yml @@ -59,7 +59,7 @@ runs: # needs, and Nix reads it from the object database instead of re-copying the # checked-out working tree the way path:$CFLC_INPUT_PATH would. See # comment-flake-lock-changelog's README's 'Disk space' section. - - uses: mdarocha/comment-flake-lock-changelog@5b367a28257ccae64ac9e4bacca51e2549534a14 # claude/build-filter-result-cache + - uses: mdarocha/comment-flake-lock-changelog@8b4c27fe668512a2552dd51ef7f4f73ee36f5a5b # claude/build-filter-result-cache if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From 12e66992d824c4d6e21e48f0c76a05b1b10c063d Mon Sep 17 00:00:00 2001 From: "robo-intern[bot]" Date: Fri, 24 Jul 2026 19:50:10 +0000 Subject: [PATCH 19/21] Bump comment-flake-lock-changelog testing pin to b055b0e Picks up the revert of the shallow-fetch experiment back to a full blobless clone, after two live failures against real nixpkgs. --- action.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/action.yml b/action.yml index 56d873c..26e135e 100644 --- a/action.yml +++ b/action.yml @@ -59,7 +59,7 @@ runs: # needs, and Nix reads it from the object database instead of re-copying the # checked-out working tree the way path:$CFLC_INPUT_PATH would. See # comment-flake-lock-changelog's README's 'Disk space' section. - - uses: mdarocha/comment-flake-lock-changelog@8b4c27fe668512a2552dd51ef7f4f73ee36f5a5b # claude/build-filter-result-cache + - uses: mdarocha/comment-flake-lock-changelog@b055b0e78a6cf6d63b351d20e680a2d98d9d54b3 # claude/build-filter-result-cache if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From 9c1448e774dbdb4e7fc896f14b24334e4da1925c Mon Sep 17 00:00:00 2001 From: mdarocha Date: Tue, 11 Aug 2026 21:25:06 +0000 Subject: [PATCH 20/21] fix: bump comment-flake-lock-changelog pin to main (fixes CI-killing 404) The old pin (b055b0e, claude/build-filter-result-cache) predates comment-flake-lock-changelog#336: compareCommits() threw uncaught on a 404 from the GitHub compare API (routine when an upstream input revision, e.g. a fast-moving nixpkgs commit, gets garbage-collected or rewritten away), killing this whole composite step after build-filter had already finished its bisection. That's exactly what broke mdarocha/pondinfra#141's Deploy check. main also picked up build-filter-gc's targeted fetch instead of a full clone, a working GitHub Actions cache key for compareCommits results, and build-filter bisection-result caching since this branch's pin was last bumped. --- action.yml | 38 ++++++++++++++++---------------------- 1 file changed, 16 insertions(+), 22 deletions(-) diff --git a/action.yml b/action.yml index 26e135e..b4482f2 100644 --- a/action.yml +++ b/action.yml @@ -37,29 +37,23 @@ runs: shell: bash run: bash "${{ github.action_path }}/scripts/setup-nixconfig.sh" - # Runs here - before the cache is restored - rather than near the end of the - # steps below, so its build-filter-gc: true is actually safe to use: the store - # is still essentially empty at this point (just the Nix installation itself), - # so nix store gc between build-filter's builds can't collect away anything the - # rest of this job needs. build-filter's own README warns that running this - # after a cache restore risks gc deleting the very cache that was just restored. - # TEMPORARY (testing): pinned to comment-flake-lock-changelog main instead of - # a release, to exercise its unreleased `build-filter`/`build-filter-gc` - # features via the changelog-filter input. See comment-flake-lock-changelog#301. - # Revert to a tagged release once those features ship. + # Runs here, before the cache is restored, so build-filter-gc: true is safe: the + # store is still essentially empty (just the Nix installation), so gc between + # build-filter's builds can't collect away anything the rest of the job needs. + # Running this after cache restore would risk gc'ing the cache that was just + # restored. # - # NOTE: build-filter-skip-checkout was tried here too, paired with the - # changelog-filter command using git+file://$CFLC_INPUT_PATH?rev=$CFLC_INPUT_REV, - # but it doesn't work: Nix's git fetcher is built on libgit2, which can't - # lazily fetch missing blobs from the blobless clone's promisor remote the way - # `git checkout` (the real git CLI) can, so skipping that checkout makes Nix - # fail with "object not found" the moment it needs a blob nothing ever fetched. - # The changelog-filter command still uses git+file://...?rev=... below, just - # without skip-checkout - the checkout still runs and fetches what libgit2 - # needs, and Nix reads it from the object database instead of re-copying the - # checked-out working tree the way path:$CFLC_INPUT_PATH would. See - # comment-flake-lock-changelog's README's 'Disk space' section. - - uses: mdarocha/comment-flake-lock-changelog@b055b0e78a6cf6d63b351d20e680a2d98d9d54b3 # claude/build-filter-result-cache + # Pinned to main rather than a tagged release: build-filter/build-filter-gc are + # still pre-release (tracked in comment-flake-lock-changelog#301). Bump this SHA + # when main moves; switch to a version tag once a release ships. + # + # build-filter-skip-checkout doesn't work: Nix's git fetcher (libgit2) can't + # lazily fetch blobs from a blobless clone's promisor remote the way `git + # checkout` can, so it fails with "object not found" once it needs one. Don't + # re-add it. `git+file://...?rev=...` (used below) is still worth it on its own: + # it reads the commit straight from the object database the checkout already + # populated, instead of re-hashing the working tree the way `path:` would. + - uses: mdarocha/comment-flake-lock-changelog@8f9c930732a4ac2b1a08ade42a0e8da7e40cb442 # main if: github.event_name == 'pull_request' with: pull-request-number: ${{ github.event.pull_request.number }} From 0da206f3c6d3ab3ec2893074f7c68f1b03fd3d5c Mon Sep 17 00:00:00 2001 From: mdarocha Date: Tue, 11 Aug 2026 21:35:16 +0000 Subject: [PATCH 21/21] perf: skip redundant pre-save Cache API call on exact primary-key hit cache-stats-capture.sh always queried the Cache API for the primary key's pre-save size, but cache-stats-report.sh's save section never consults that value when restore already hit the primary key exactly - it short-circuits straight to "save skipped" instead. Skip the query in that case, using the already-known restore outcome from cache-meta.env. --- scripts/cache-stats-capture.sh | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/scripts/cache-stats-capture.sh b/scripts/cache-stats-capture.sh index 2eefbc9..9e34bb0 100755 --- a/scripts/cache-stats-capture.sh +++ b/scripts/cache-stats-capture.sh @@ -37,9 +37,27 @@ if ! nix --extra-experimental-features nix-command path-info --all --json 2>/dev : > "$state_dir/now.tsv" fi -pre_save_size="$(nms_cache_size_bytes "${CACHE_PRIMARY_KEY:-}")" -existed=false -[ -n "$pre_save_size" ] && existed=true +# When restore already hit the primary key exactly, the report's save section +# short-circuits without ever consulting CACHE_PRIMARY_EXISTED_PRE_SAVE or +# CACHE_PRIMARY_SIZE_PRE_SAVE (see cache-stats-report.sh), so querying the API +# here in that case would just be a wasted round trip. cache-meta.env, from +# the earlier "post" snapshot, is how we know the restore outcome. +hit_primary="" +meta_file="$state_dir/cache-meta.env" +if [ -f "$meta_file" ]; then + # shellcheck source=/dev/null + . "$meta_file" + hit_primary="${CACHE_HIT_PRIMARY_KEY:-}" +fi + +if [ "$hit_primary" = "true" ]; then + existed=true + pre_save_size="" +else + pre_save_size="$(nms_cache_size_bytes "${CACHE_PRIMARY_KEY:-}")" + existed=false + [ -n "$pre_save_size" ] && existed=true +fi { printf 'CACHE_PRIMARY_EXISTED_PRE_SAVE=%s\n' "$existed" printf 'CACHE_PRIMARY_SIZE_PRE_SAVE=%s\n' "$pre_save_size"