feat(export): persistent on-disk helmTemplate cache#4
Merged
Conversation
NickAnge
previously approved these changes
Jun 5, 2026
Add an experimental, opt-in `--helm-cache` flag to `rtk export` that maintains a `helm-cache/` directory inside the output dir, persisting helmTemplate results across runs and environments. Builds on the process-global in-memory Helm cache: the in-memory value is now the manifested JSON of the rendered resource map (not raw YAML), so cache hits skip both the helm subprocess and YAML parsing - a hit is a single serde_json::from_str into a Val. The cache key now also covers nameFormat and the chart's Chart.yaml contents. The disk cache is global and loaded/written exactly once per run: before the parallel loop all entries are preloaded into the in-memory cache and the directory is cleared; after the loop only the keys touched this run are written back, pruning stale entries. A process-global touched set is used because a single main.jsonnet can expand into many environments across worker threads. Also add RTK_HELM_DISABLE_MEMOIZATION as a benchmarking escape hatch that bypasses the in-memory cache so every helmTemplate call invokes helm. Co-authored-by: Cursor <cursoragent@cursor.com>
The delete_opts test constructs ExportOpts with all fields explicitly (no ..Default::default()), so it needs the new helm_cache field. Co-authored-by: Cursor <cursoragent@cursor.com>
Add a self-contained uv script and fixture that measure the three Helm templating cache regimes via hyperfine: no-memoization (RTK_HELM_DISABLE_MEMOIZATION=1), the default in-memory cache, and a warm --helm-cache. The fixture renders one local chart across 60 inline environments with identical parameters, mirroring a fleet that renders the same chart across many clusters. Co-authored-by: Cursor <cursoragent@cursor.com>
a340dd5 to
8c49b46
Compare
Benchmark Results
Full results available in workflow artifacts. Benchmark run on commit |
Wire the helmTemplate caching benchmark into the CI benchmark matrix via a new "static" mode in run-benchmark.py (pre-existing fixtures dir + explicit export commands, rtk-vs-tk-vs-base, no mock k8s server). Replaces the standalone helm-template-benchmark.py, which produced no table row. Also adds unit tests for the on-disk helm cache and helm cache key. Co-authored-by: Cursor <cursoragent@cursor.com>
Expand the benchmark chart to 15 templates (~90 resources) and concentrate cost in CPU-bound rendering (chained sha256sum) so the cached render dominates. Output is kept moderate since large YAML mainly adds per-environment serialization, where Go's yaml is faster and erodes the cache's advantage. Co-authored-by: Cursor <cursoragent@cursor.com>
NickAnge
approved these changes
Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Exporting many environments that render the same Helm charts pays the helm-rendering cost on every run. Add an experimental, opt-in
--helm-cacheflag tortk exportthat keeps ahelm-cache/directory inside the output dir and reuses helmTemplate results across runs and environments.helmsubprocess and YAML parsing: the cached value is the manifested JSON of the rendered resource map, so a hit is a single deserialize into aValnameFormat, and the chart'sChart.yamlcontents (so a vendored version bump invalidates stale entries)RTK_HELM_DISABLE_MEMOIZATIONto bypass the in-memory cache entirely (helm runs on every call) for benchmarkingContext
Builds on the process-global in-memory Helm cache. That cache now stores the post-parse JSON projection instead of raw YAML, so hits no longer re-parse YAML, and it is shared across worker threads — a chart rendered for one environment is reused by every other. The on-disk cache is global and loaded/written exactly once per run: before the parallel loop every entry is preloaded into memory and the directory is cleared; after the loop only the keys touched this run are written back, which prunes stale entries. A process-global touched set is used because a single
main.jsonnetcan expand into many environments across worker threads (an earlier per-environment design raced on the shared directory).Gated behind
--helm-cache(off by default) so the metadata directory never appears in normal exports or golden-test output.Stacked on #3.
Made with Cursor