ci: cache pip, npm, and JS parser node_modules to speed up Windows runs#40
Merged
joshbouncesecurity merged 4 commits intomasterfrom Apr 19, 2026
Merged
ci: cache pip, npm, and JS parser node_modules to speed up Windows runs#40joshbouncesecurity merged 4 commits intomasterfrom
joshbouncesecurity merged 4 commits intomasterfrom
Conversation
Windows CI was 3-5x slower than Linux (Go job: 3m23s vs 1m13s) largely because Defender scanning + slow NTFS metadata ops on cold toolchain extraction dominate wall time, and the workflow was redownloading/reinstalling all deps on every run. Three changes: - actions/setup-python: add `cache: pip` keyed on pyproject.toml. - actions/setup-node: add `cache: npm` keyed on package-lock.json. - actions/cache: cache parsers/javascript/node_modules directly keyed on the lockfile hash, and skip `npm install` entirely on cache hit. Also switch to `npm ci` for the miss path (stricter, deterministic). setup-go already caches the build + module cache via cache-dependency-path, so no change there. Expected: cache-hit runs should drop the Windows Go job from ~3m20s to ~1m30s and save ~30-40s on each Python job. First run on a new lockfile hash repopulates the cache. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joshbouncesecurity
added a commit
that referenced
this pull request
Apr 19, 2026
Adapted from master branch's ci/cache-deps (fork PR #40) for this branch's v6-actions, requirements.txt-based workflow. - actions/setup-python: add `cache: pip` keyed on requirements.txt. - actions/setup-node: add `cache: npm` keyed on package-lock.json. - actions/cache@v4: cache parsers/javascript/node_modules directly keyed on the lockfile hash; skip `npm install` entirely on cache hit. Miss path uses `npm ci` for determinism. setup-go already caches the build + module dirs via cache-dependency-path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add-MpPreference -ExclusionPath on each Windows job to skip real-time scanning over the checkout dir, Go build cache (~AppData\Local\go-build), Go module cache (~\go\pkg\mod), and npm's cache dirs. Real-time scanning of compiler-generated intermediates and extracted modules was the largest remaining slowdown on Windows runners after dep caching. Also drops the retrigger-comment scaffolding from the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Defender exclusion step didn't produce a reliable improvement: Go Windows went 2m55s → 3m48s with it added (vs 2m55s without), which is either noise or evidence that Add-MpPreference is silently no-op'd on GitHub-hosted runners by tamper protection. Dropping it keeps the workflow smaller and leaves the caching wins intact. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Windows CI is ~3–5× slower than Linux on this repo (Go job: 3m23s vs 1m13s on the last master run) because (a) the runner cold-extracts toolchains on every run, (b) Defender real-time scanning taxes NTFS metadata ops, and (c) nothing was cached between runs except the Go module/build dirs.
Three cache additions:
actions/setup-python@v5:cache: pipkeyed onlibs/openant-core/pyproject.toml.actions/setup-node@v5:cache: npmkeyed onlibs/openant-core/parsers/javascript/package-lock.json.actions/cache@v4: cachelibs/openant-core/parsers/javascript/node_modulesdirectly, keyed on the lockfile hash. Thenpm installstep is gated byif: cache-hit != 'true'so warm runs skip it entirely. Miss path usesnpm cifor determinism.setup-go@v5already caches~/go/pkg/mod+ the build cache viacache-dependency-path, so no change there.Test plan
npm cimiss-path still produces a workingnode_modules/(verified implicitly by existing JS parser tests).🤖 Generated with Claude Code