Skip to content

feat(jsonnet): preserve hidden fields in rtkMemoize#7

Closed
julienduchesne wants to merge 1 commit into
masterfrom
julienduchesne/rtk-memoize-hidden-fields
Closed

feat(jsonnet): preserve hidden fields in rtkMemoize#7
julienduchesne wants to merge 1 commit into
masterfrom
julienduchesne/rtk-memoize-hidden-fields

Conversation

@julienduchesne

Copy link
Copy Markdown
Collaborator

Summary

rtkMemoize cached values as a JSON string so it could share them across worker threads, which silently dropped hidden (::) fields — so it rejected any value containing one. Swap that JSON round-trip for a thread-portable OwnedVal snapshot that mirrors the jsonnet value model, so hidden fields survive memoization.

  • Snapshot records per-field visibility and is rebuilt into a fresh Val (with :: restored) in each consuming thread; cache hits clone an Arc instead of reparsing JSON.
  • Functions still can't be memoized, but instead of failing on the first one the error lists the path of every function found (e.g. spec.template.withName, <root>, items[0]).

Context

Val is Rc/Cc-based and bound to the evaluating thread, so the cross-worker cache must hold a Send/Sync value and rebuild per thread. JSON was the easy portable form but lossy. OwnedVal is an owned deep copy (scalars/arrays/objects + visibility) that stays portable without losing hidden data. Functions remain out of scope: a FuncVal captures a closure Context (free vars, self/super, std, imports) plus Rust builtins, none of which reduce to thread-independent data.

One behavioral note: snapshotting forces every field, including hidden ones, so a hidden field that errors now fails the memoize call even if it is never used.

Made with Cursor

Store memoized values as a thread-portable OwnedVal snapshot instead of a
lossy JSON string. Hidden (::) object fields now round-trip through the
cross-worker cache, where previously any hidden field was rejected.

Functions still cannot be memoized (they capture a thread-local closure
context plus Rust builtins), but the error now lists the path of every
function found in the value rather than failing on the first one.

Co-authored-by: Cursor <cursoragent@cursor.com>
@julienduchesne julienduchesne requested a review from a team as a code owner June 9, 2026 16:59
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

Benchmark Results

Benchmark Test vs tk vs base
Diff cluster_scoped 29.32x faster equal
Diff configmap_unchanged 29.51x faster equal
Diff deployment_nested_changes 27.99x faster equal
Env List Single Inline Directory (--json) 47.15x faster equal
Env List Single Inline File (--json) 46.99x faster equal
Env List All Environments (--json) 83.23x faster equal
Eval Single Static Environment 15.86x faster equal
Eval Inline Environment File 19.85x faster equal
Eval Eval Expression - Deeply Nested Path 16.04x faster equal
Export (Full) Single Static Environment 6.94x faster equal
Export (Full) Single Inline Environment 21.35x faster equal
Export (Full) All Environments 19.05x faster equal
Export (Replace) Single Static Environment 5.65x faster equal
Export (Replace) Single Inline Environment 21.24x faster equal
Export (Replace) All Environments 27.79x faster equal
Helm Template Full Export 1.4x faster equal
Tool Importers Count Single Lib Directory (Non-Recursive) 2.39x faster equal
Tool Importers Count Single Lib Directory (Recursive) 2.25x faster equal
Tool Importers Count Entire Lib Directory (Non-Recursive) 1.36x faster equal
Tool Importers Count Entire Lib Directory (Recursive) 1.11x faster equal
Tool Importers Global Lib File 3.17x faster equal
Tool Importers Single Env-Specific Lib File 1.68x faster equal
Tool Importers All Env-Specific Lib Files 5.18x faster equal
Tool Importers All Lib Files 5.21x faster equal
Tool Importers All Jsonnet Files 8.83x faster equal
Tool Imports Single Static Environment 3.56x faster equal
Tool Imports Single Inline Environment File 3.48x faster equal
Validate Manifests All Validations - equal
Validate Manifests Manifest Tests Only - equal
Validate Manifests Namespace Test Only - equal

Full results available in workflow artifacts.

Benchmark run on commit 5eed8a9bfed573af618c9692555151163c2d5e22

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant