Skip to content

feat(export): support s3 and configurable location for helm cache#5

Closed
julienduchesne wants to merge 1 commit into
masterfrom
julienduchesne/helm-cache-s3
Closed

feat(export): support s3 and configurable location for helm cache#5
julienduchesne wants to merge 1 commit into
masterfrom
julienduchesne/helm-cache-s3

Conversation

@julienduchesne

Copy link
Copy Markdown
Collaborator

Summary

Extend the experimental helmTemplate export cache so it can live somewhere other than the output directory — including S3 — and so callers can decide how strict cache failures should be.

  • Replace --helm-cache-dir with --helm-cache-path, which accepts a local path, a file:// URL, or an s3://bucket/prefix URL. Setting it enables the cache and overrides the default <output_dir>/helm-cache.
  • Add --helm-cache-on-error=warn|fail: warn logs and continues (the export still succeeds without a persisted cache), fail aborts the export.
  • Abstract the cache behind a CacheLocation backend. The S3 backend lists and fetches entries into the in-memory cache on load, then puts the entries touched this run and prunes stale objects under the prefix on save.

Context

The cache load/save run once per export, single-threaded outside the rayon pool, so the S3 calls are driven by a private current-thread tokio runtime. S3 credentials and region come from the standard AWS default chain (env, profile, IMDS).

Note: like the local backend, an export that makes no helmTemplate calls prunes the whole prefix (nothing touched), so pointing --helm-cache-path at a shared S3 prefix and running a helm-free export will clear it.

Made with Cursor

Add an `s3://` backend and a configurable location for the experimental
helmTemplate export cache.

- Replace `--helm-cache-dir` with `--helm-cache-path`, accepting a local
  path, `file://` URL, or `s3://bucket/prefix` URL; setting it enables the
  cache and overrides the default `<output_dir>/helm-cache`
- Add `--helm-cache-on-error=warn|fail` to choose whether load/save
  failures warn and continue or abort the export
- Abstract the cache behind a CacheLocation backend (local dir or S3
  prefix); the S3 backend lists/gets entries on load and puts touched
  entries plus prunes stale ones on save

Co-authored-by: Cursor <cursoragent@cursor.com>
@julienduchesne julienduchesne requested a review from a team as a code owner June 8, 2026 19:56
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

Benchmark Results

Benchmark Test vs tk vs base
Diff cluster_scoped 1.43x faster 39.33x slower
Diff configmap_unchanged 0.79x faster 38.51x slower
Diff deployment_nested_changes 0.81x faster 37.73x slower
Env List Single Inline Directory (--json) 50.06x faster equal
Env List Single Inline File (--json) 48.83x faster equal
Env List All Environments (--json) 87.17x faster equal
Eval Single Static Environment 14.96x faster 1.04x slower
Eval Inline Environment File 19.18x faster equal
Eval Eval Expression - Deeply Nested Path 14.11x faster equal
Export (Full) Single Static Environment 6.47x faster equal
Export (Full) Single Inline Environment 20.63x faster equal
Export (Full) All Environments 19.3x faster equal
Export (Replace) Single Static Environment 5.6x faster equal
Export (Replace) Single Inline Environment 21.91x faster equal
Export (Replace) All Environments 28.19x faster equal
Helm Template Full Export 1.36x faster equal
Tool Importers Count Single Lib Directory (Non-Recursive) 2.18x faster equal
Tool Importers Count Single Lib Directory (Recursive) 2.21x faster equal
Tool Importers Count Entire Lib Directory (Non-Recursive) 1.32x faster equal
Tool Importers Count Entire Lib Directory (Recursive) 1.16x faster equal
Tool Importers Global Lib File 3.26x faster equal
Tool Importers Single Env-Specific Lib File 1.67x faster equal
Tool Importers All Env-Specific Lib Files 5.12x faster equal
Tool Importers All Lib Files 5.17x faster equal
Tool Importers All Jsonnet Files 8.83x faster equal
Tool Imports Single Static Environment 3.11x faster equal
Tool Imports Single Inline Environment File 3.12x faster equal
Validate Manifests All Validations - equal
Validate Manifests Manifest Tests Only - equal
Validate Manifests Namespace Test Only - equal

Full results available in workflow artifacts.

Benchmark run on commit ec458ae95eec4f10724c55b41b91f593d067c49d

@julienduchesne julienduchesne deleted the julienduchesne/helm-cache-s3 branch June 9, 2026 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant