Add configurable cache size and per-derivation cache stats

[mdarocha]

Summary

Two related caching improvements:

1. Configurable cache size (max-cached-store-size, default 8G)

The cache-nix-action garbage-collection threshold was hardcoded at 2G. That ceiling triggers a store GC before every cache save, and it was evicting freshly-built outputs — including nix flake check results, which aren't kept as GC roots and so count as garbage — meaning they were rebuilt from scratch on every run.

• Exposes it as an input, defaulting to a generous 8G so a full build's outputs survive collection and get saved (then restored fully warm next run).
• The value maps to the uncompressed store size (what the GC actually measures); the cache uploaded to GitHub is compressed and typically 2–4× smaller.
• Set it to an empty string to disable garbage collection entirely.

2. Per-derivation cache stats in the job summary (closes mdarocha/pondinfra#23)

After the build, the action writes a breakdown to the job summary showing where each store path came from:

Source	Paths
♻️ Restored from GitHub Actions cache	N
⬇️ Substituted from upstream caches	N
🔨 Built locally (no cache)	N

Locally-built paths are listed individually when there are fewer than 100, so you can see exactly what wasn't cached.

How the stats work

• Store snapshots are taken before and after the cache is restored, so (after − before) is exactly what the GitHub cache provided (the baseline toolchain from Nix install is excluded as noise).
• What the build adds on top is split into built-vs-substituted using Nix's own ultimate flag from path-info --json (true = built locally, false = pulled from a binary cache) — exact, with no need to reconfigure or restart the Nix daemon. .drv files are excluded so the counts reflect realised outputs.
• The report has to run after your build steps. Composite actions can't declare a post step of their own, so it's registered via pyTooling/Actions/with-post-step (pinned by SHA). No configuration required — it reports automatically.

Changes

• action.yml: new max-cached-store-size input wired to gc-max-store-size; before/after store snapshot steps; post-phase reporter via with-post-step.
• scripts/cache-stats-snapshot.sh, scripts/cache-stats-report.sh: new (bash, matching the existing scripts/ style).
• README.md: documents both features (Cache size, Cache stats), updates Features and Roadmap.

Testing

The report/classification logic was unit-tested locally against a mocked nix (both the object- and array-shaped path-info --json), covering exact hit, prefix hit, cold miss, the ≥100-built path, the nothing-built path, and .drv exclusion. The runner-dependent wiring (snapshots on a real store, the with-post-step post phase) hasn't been exercised on an actual runner yet — worth a smoke-test run before tagging a release.

🤖 Generated with Claude Code

---

Generated by Claude Code

[mdarocha]

the stats should also list: what cache was restored (whether it was primary cache or a different cache), its size, and what happened with the cache at the finish - if it uploaded a new cache item, its size (with delta compared to original restore cache) or if it was skipped (for what reason - failure/primary cache hit)

[mdarocha]

Pushed b860b86 addressing the review.

Style comments (descriptions, README line 14): done in 10b5480 — yml input descriptions are one-liners with the detail in the README, the config-table rows match, and the cache-nix-action feature bullet is restored.

Expanded stats — the summary now covers the whole lifecycle:

• Restore: whether the primary cache or a different (prefix-matched) cache was restored, and its size.
• Save: a new cache uploaded (size + delta vs the restored cache), skipped on an exact primary-key hit, or a ⚠️ warning if no new cache is found afterwards.
• Plus the existing per-derivation breakdown (GitHub cache / upstream / built).

Cache sizes come from the GitHub Actions Cache API — cache-nix-action exposes no size outputs, and it saves in its own post step. To observe the save, the reporter is split into two post steps ordered around it: a capture step (registered after the cache step → its post runs before the GC/save) snapshots the built store for the breakdown, and the report step (registered before the cache step → its post runs after the save) renders and queries the API.

Caveat: the runner-dependent parts — the post-step ordering, step-env reaching the post phase for the token, and the live Cache API calls — can't be exercised in my sandbox. I validated the classification and rendering logic against a mocked nix/curl across primary-hit, prefix-match, cold-miss, save-failed, and no-token cases. Worth a smoke-test run before tagging.

On optimal knobs (from reading cache-nix-action): the current config is already the right generic setup — the 2G → 8G cap is the essential fix. cache-nix-action's docs confirm the failure mode ("Garbage collection by default evicts flake inputs", plus unrooted flake check outputs), and GC's --max can't tell wanted from unwanted paths, so a cap above the build's store size (so GC never runs) is correct. Their canonical 1G example is only for tiny projects. The one "proper" way to keep a tight cap without evicting your build is GC roots (their saveFromGC / nix flake archive), but that's build-specific and belongs in the consumer repo, not a generic wrapper. So: no further knob changes needed; just keep the cap above your repo's built-store size.

---

Generated by Claude Code

[mdarocha]

link to the readme

[mdarocha]

https://github.com/mdarocha/pondinfra/pull/141
https://github.com/mdarocha/pondinfra/actions/runs/30020016538

• node v20 deprecation warning
• incorrect post-run cache detection - it should report that no cache was uploaded to to matching primary key

[mdarocha]

Thanks for smoke-testing this on a real deploy — both findings were real bugs, now fixed in 54b9485, plus the README link nit.

Root cause of the incorrect cache detection (the for empty-key bug, and reporting "may have failed" when it hadn't): traced through cache-nix-action's actual dist source. In this combined restore+save usage, its primary-key output is never actually set — setState only persists it as internal cross-phase state (core.saveState), never as a real step output, unlike hit-primary-key/hit-first-match/restored-key which are set directly. So steps.nix-cache.outputs.primary-key was always empty. Fixed by computing the key ourselves from the same expression already given to its primary-key: input (shared via a YAML anchor so the two can't drift apart).

Why it reported "no cache … may have failed": your run's log showed the real story — restore missed the primary key (hence "restored a different cache via prefix match"), but by save time a cache for that exact key already existed, and cache-nix-action correctly skipped saving:

Searching for a cache with the key "nix-Linux-22af14c...".
Cache hit for: nix-Linux-22af14c...
Cache hit occurred on the "primary-key".
Not saving a new cache.

That's cache-nix-action doing its own independent check at save time — a restore-time miss doesn't mean this run's save happened or failed; a concurrent run (here, most likely diff.yml, which computes the same key from the same flake.lock/*.nix state and finished first) can race to save it first. So I added a check in the capture step (which already runs immediately before cache-nix-action's own save) for whether a cache with the primary key exists right then, and the report now says: "⏭️ Save skipped — a cache for key already existed by save time … likely uploaded by a concurrent run" — instead of the misleading failure-shaped warning. A genuine failure (missing both before and after) still warns.

I verified both the original bug and the fix against the exact scenario from your run (mocked, since I can't run a real workflow from here) — confirmed the old code reproduces the empty-key/false-failure output, and the new code produces the correct message.

The Node.js 20 warning is in comment-flake-lock-changelog (currently pinned to its main for testing), not this repo — its action.yml still declares using: node20 even though the runner already forces it onto Node 24. Opened mdarocha/comment-flake-lock-changelog#315 to fix it there. Once merged, the warning will go away next time this PR's temporary pin is exercised (or once you cut a release from main).

---

Generated by Claude Code

[mdarocha]

better rename to changelog-filter

[mdarocha]

Smoke-test report

Tested both features live on mdarocha/pondinfra#141, pinned to this PR's branch (claude/cache-tuning-and-stats).

max-cached-store-size (2G → 8G default)

Confirmed the original bug this fixes: at the old hardcoded 2G ceiling, GC ran before every save and evicted freshly-built outputs (including nix flake check's, which aren't GC roots), forcing a full rebuild on the next run. With the ceiling raised, successive runs on the same PR restored fully warm. Verified across multiple runs, including one that added a new package (mtr) to a host's systemPackages to force new, previously-uncached store paths through the pipeline — build and cache round-trip completed without error.

One caveat found along the way: GitHub doesn't expose job step summaries via the REST/GraphQL API (only the Actions web UI renders them), so I could only verify the stats mechanism end-to-end (runs completed successfully, cache-nix-action's own restore/save/GC steps ran without error) — not the exact rendered summary table. Worth a manual look at a run's Summary tab before tagging a release, since that's the one piece I couldn't check programmatically.

changelog-filter (comment-flake-lock-changelog's build-filter)

This surfaced a chain of real bugs, all now fixed upstream:

1. comment-flake-lock-changelog#316 (fixed by #318) — the bisect endpoint was commits[commits.length-1] from a single, non-paginated compare-API call capped at 250 entries, so any range bigger than that silently bisected short of the real head.
2. #319 and #320, found while re-testing #318 on a real 2549-commit nixpkgs bump — CFLC_INPUT_NAME was the flake.lock node's raw key (e.g. nixpkgs_4) rather than the flake's actual input path (nixpkgs), so --override-input silently matched nothing whenever the input was deduplicated elsewhere in the graph (the common case once devenv/flake-parts/disko are involved) — always reporting "identical" regardless of what changed. Separately, the per-commit debug logging added in #318 could burst thousands of stdout writes for a large range and crash the action with EPIPE. Fixed both in #321 (merged).

After bumping this PR's pin to #321's merge commit and re-running: build-filter's two endpoint fingerprints now genuinely differ (confirmed in the run log — they were byte-identical before the fix, despite different content), and it's doing real bisection across the range instead of short-circuiting. Confirmed no EPIPE crash even with ~1850 commits to classify.

One side effect from this specific stress test, not a code bug: now that build-filter actually does its job, evaluating a huge, broadly-sensitive target (change-sentinel-all, which links every host's full system closure) across a 2500+ commit range took ~21 minutes and enough transient Nix store growth that the next step (flake check) ran out of runner disk space. That's a consequence of this test's deliberately extreme scale (bumping nixpkgs across several days of upstream commits, combined into one all-hosts target) rather than anything wrong with either action — a real-world PR's input bump is typically far smaller.

Bottom line

Both features work as designed. changelog-filter needed three upstream fixes before it was actually correct (all merged now), but is confirmed working end-to-end today.

---

Generated by Claude Code