feat(export): support s3 and configurable location for helm cache#5
Closed
julienduchesne wants to merge 1 commit into
Closed
feat(export): support s3 and configurable location for helm cache#5julienduchesne wants to merge 1 commit into
julienduchesne wants to merge 1 commit into
Conversation
Add an `s3://` backend and a configurable location for the experimental helmTemplate export cache. - Replace `--helm-cache-dir` with `--helm-cache-path`, accepting a local path, `file://` URL, or `s3://bucket/prefix` URL; setting it enables the cache and overrides the default `<output_dir>/helm-cache` - Add `--helm-cache-on-error=warn|fail` to choose whether load/save failures warn and continue or abort the export - Abstract the cache behind a CacheLocation backend (local dir or S3 prefix); the S3 backend lists/gets entries on load and puts touched entries plus prunes stale ones on save Co-authored-by: Cursor <cursoragent@cursor.com>
Benchmark Results
Full results available in workflow artifacts. Benchmark run on commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extend the experimental helmTemplate export cache so it can live somewhere other than the output directory — including S3 — and so callers can decide how strict cache failures should be.
--helm-cache-dirwith--helm-cache-path, which accepts a local path, afile://URL, or ans3://bucket/prefixURL. Setting it enables the cache and overrides the default<output_dir>/helm-cache.--helm-cache-on-error=warn|fail:warnlogs and continues (the export still succeeds without a persisted cache),failaborts the export.CacheLocationbackend. The S3 backend lists and fetches entries into the in-memory cache on load, then puts the entries touched this run and prunes stale objects under the prefix on save.Context
The cache load/save run once per export, single-threaded outside the rayon pool, so the S3 calls are driven by a private current-thread tokio runtime. S3 credentials and region come from the standard AWS default chain (env, profile, IMDS).
Note: like the local backend, an export that makes no helmTemplate calls prunes the whole prefix (nothing touched), so pointing
--helm-cache-pathat a shared S3 prefix and running a helm-free export will clear it.Made with Cursor