Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ gen-scope is generic. It has no knowledge of NixOS, aspects, policies, or system
- [API Reference](#api-reference)
- [eval](#eval)
- [evalDebug](#evaldebug)
- [evalWarm](#evalwarm)
- [recordedDeps](#recordeddeps)
- [buildNodes](#buildnodes)
- [Algebraic Graph Construction](#algebraic-graph-construction)
- [Attribute Combinators](#attribute-combinators)
Expand Down Expand Up @@ -221,6 +223,40 @@ Returns `{ node, get, allNodes, allNodesWhere, subtreeOf, nodesOfType }`:

Same interface as `eval`. Provides structured cycle traces instead of Nix's opaque "infinite recursion." Trade-off: defeats memoization. Use for diagnosing cycles only.

### `evalWarm`

```nix
evalWarm {
roots; # { id = { id, type, parent, decls }; }
attributes; # { attrName = self: id: value; }
parseParent ? null; # id → parentId | null
priorResults; # { id = { attrName = cachedValue; }; }
isClean; # id → bool
}
```

Warm-cache variant of `eval` for incremental re-evaluation. A leaf attribute of a **clean** node (`isClean id == true`) whose value is present in `priorResults` is served from the cache **without forcing its compute function**; everything else evaluates cold. `children`/`derived-children` are **never** served warm — tree structure is always recomputed, so a dirty descendant stays reachable through freshly-materialized parents. Returns the same shape as `eval`.

With `eval`'s defaults (`priorResults = {}`, `isClean = _: false`) the warm branch never fires, so `eval` and warm-off `evalWarm` are byte-identical — they share a single code path.

```nix
result = engine.evalWarm {
inherit roots attributes;
priorResults = { "host:web" = { region = "us-east"; }; }; # from a prior eval
isClean = id: id != "host:db"; # only db changed
};
result.get "host:web" "region" # "us-east" — served warm, compute fn not forced
result.get "host:db" "region" # recomputed (db is dirty)
```

### `recordedDeps`

```nix
recordedDeps { declaredEdges } id # → [id]
```

First-class projection of a consumer's **declared** read-edges: it simply applies `declaredEdges id`. Pure and memo-free — it never runs through `get`. The *dynamic* read-set (the attributes a node actually `self.get`s) is only recoverable via `evalDebug`'s fresh-`self`-per-`get`, which defeats memoization; there is no pure, memo-preserving way to capture it, so the declared edges are the inspectable contract. Useful for incremental consumers that need an explicit dependency edge set (e.g. driving a rebuild).

### `buildNodes`

```nix
Expand Down Expand Up @@ -317,10 +353,12 @@ collectionAttr { traverse; extract; combine ? a: b: a ++ b; filter ? _: true; }

Traverse modes: `"imports"`, `"children"`, `"siblings"`, `"ancestors"`, `"neron"`, `"label:<name>"`, or custom function.

**`"neron"` traverse mode** — Specificity-ordered collection following D > I > P (declaration, import, parent) priority. Unlike `query`, which returns a single shadowed result, `"neron"` returns all contributions from reachable scopes in specificity order, suitable for fold-based composition (e.g., collecting all modules, merging config fragments).
**`"neron"` traverse mode** — Specificity-ordered collection following D over I over P (local shadows import shadows parent) priority. Unlike `query`, which returns a single shadowed result, `"neron"` returns all contributions from reachable scopes in specificity order, suitable for fold-based composition (e.g., collecting all modules, merging config fragments).

Properties: cycle-safe via seen-set tracking, diamond-safe deduplication (each scope visited at most once), recursive parent resolution. Traversal order: self, then unseen imports, then parent — mirroring the Neron (2015) resolution calculus but collecting rather than shadowing.

The neron traversal order (self → imports → parent, imports in declaration order) is a public, stable contract: collection determinism for ordered-list channels rests on this pin plus a left fold, so changing the traversal order is a breaking change.

```nix
# Collect all config fragments from local scope, imports, and ancestors
config-modules = engine.collectionAttr {
Expand Down Expand Up @@ -409,7 +447,7 @@ just ci eval # run eval suite
just ci eval.test-basic-root-attribute # specific test
```

Requires nix-unit. 152 tests across 10 suites.
Requires nix-unit. 163 tests across 19 suites (12 test files).

## Theoretical Foundations

Expand All @@ -425,3 +463,4 @@ Requires nix-unit. 152 tests across 10 suites.
| Radul & Sussman (2009) "Art of the propagator" | **Informed by** | Monotonic convergence concept for `circular` attribute iteration; cells accepting information from multiple sources as design influence on scope graph merging |
| Van Wyk et al. (2010) "Silver: extensible AG" | **Informed by** | Forwarding concept (productions defining default attribute values via translation); collection attributes with fold operators as design influence on `collectionAttr` |
| Mokhov et al. (2018) "Build systems a la carte" | **Informed by** | Demand-driven evaluation as suspending scheduler (§4.1); Nix's lazy evaluation recognized as the scheduling mechanism — we do not build a scheduler, Nix is the scheduler |
| Acar et al. (2006) "Adaptive functional programming" | **Informed by** | Warm-cache incremental re-evaluation (`evalWarm`): reusing clean prior results and recomputing only dirty nodes; `recordedDeps` as the declared read-edge projection of a dynamic dependence graph |
278 changes: 278 additions & 0 deletions ci/tests/eval.nix
Original file line number Diff line number Diff line change
Expand Up @@ -314,4 +314,282 @@ in
];
};
};

# appended inside the same `{ ... }` the module returns, as a sibling of flake.tests."eval"
flake.tests."eval-warm" =
let
mkRoots =
decls:
lib.mapAttrs (id: d: {
inherit id;
type = "t";
parent = null;
decls = d;
}) decls;
poison = {
children = self: id: { };
boom = self: id: throw "boom-${id}";
val = self: id: (self.node id).decls.v or 0;
};
in
{
# clean + prior present ⇒ warm-served, fn NEVER forced (poison doesn't throw)
test-warm-serves-clean-no-force = {
expr =
(genScope.evalWarm {
roots = mkRoots {
n = {
v = 5;
};
};
attributes = poison;
priorResults = {
n = {
boom = "cached";
};
};
isClean = _: true;
}).get
"n"
"boom";
expected = "cached";
};
# dirty ⇒ recompute ⇒ poison fn runs ⇒ throws
test-dirty-recomputes = {
expr =
(builtins.tryEval (
(genScope.evalWarm {
roots = mkRoots {
n = {
v = 5;
};
};
attributes = poison;
priorResults = {
n = {
boom = "cached";
};
};
isClean = _: false;
}).get
"n"
"boom"
)).success;
expected = false;
};
# clean but attr absent from priorResults ⇒ falls through to fn ⇒ throws
test-warm-missing-attr-falls-through = {
expr =
(builtins.tryEval (
(genScope.evalWarm {
roots = mkRoots {
n = {
v = 5;
};
};
attributes = poison;
priorResults = {
n = { };
};
isClean = _: true;
}).get
"n"
"boom"
)).success;
expected = false;
};
# warm value is served verbatim (not the fresh computation)
test-warm-serves-prior-value = {
expr =
(genScope.evalWarm {
roots = mkRoots {
n = {
v = 5;
};
};
attributes = poison;
priorResults = {
n = {
val = 99;
};
};
isClean = _: true;
}).get
"n"
"val";
expected = 99;
};
# a dirty node reads a clean dep's WARM value
test-dirty-reads-clean-dep = {
expr =
(genScope.evalWarm {
roots = mkRoots {
a = {
base = 10;
};
b = {
base = 0;
};
};
attributes = {
children = self: id: { };
val = self: id: if id == "b" then (self.get "a" "val") + 1 else (self.node id).decls.base;
};
priorResults = {
a = {
val = 99;
};
};
isClean = id: id == "a";
}).get
"b"
"val";
expected = 100;
};
# children NEVER warm-served (stale prior children ignored; fresh structure used)
test-children-always-recomputed = {
expr =
let
attrs = {
children =
self: id:
if id == "p" then
{
c = {
id = "c";
type = "t";
parent = "p";
decls = { };
};
}
else
{ };
label = self: id: "fresh-${id}";
};
w = genScope.evalWarm {
roots = mkRoots { p = { }; };
attributes = attrs;
priorResults = {
p = {
children = {
BOGUS = { };
};
};
};
isClean = _: true;
};
in
builtins.attrNames (w.get "p" "children");
expected = [ "c" ];
};
# a dirty GRANDCHILD is reachable through freshly-recomputed children
test-dirty-grandchild-reachable = {
expr =
let
attrs = {
children =
self: id:
if id == "p" then
{
c = {
id = "c";
type = "t";
parent = "p";
decls = { };
};
}
else if id == "c" then
{
g = {
id = "g";
type = "t";
parent = "c";
decls = { };
};
}
else
{ };
label = self: id: "fresh-${id}";
};
w = genScope.evalWarm {
roots = mkRoots { p = { }; };
attributes = attrs;
priorResults = {
g = {
label = "stale-g";
};
};
isClean = id: id != "g"; # g dirty, p/c clean
};
in
w.get "g" "label";
expected = "fresh-g"; # g recomputed (dirty) AND reachable (children recomputed, never warm)
};
# per-(node,attr) granularity: clean node, attr a warm / attr b cold
test-mixed-warm-cold = {
expr =
let
w = genScope.evalWarm {
roots = mkRoots { n = { }; };
attributes = {
children = self: id: { };
a = self: id: "fresh-a";
b = self: id: "fresh-b";
};
priorResults = {
n = {
a = "cached-a";
};
}; # has a, not b
isClean = _: true;
};
in
[
(w.get "n" "a")
(w.get "n" "b")
];
expected = [
"cached-a"
"fresh-b"
];
};
# evalWarm with warm-off defaults == eval
test-evalWarm-defaults-equal-eval = {
expr =
(genScope.evalWarm {
roots = mkRoots {
n = {
v = 7;
};
};
attributes = poison;
priorResults = { };
isClean = _: false;
}).get
"n"
"val";
expected =
(genScope.eval {
roots = mkRoots {
n = {
v = 7;
};
};
attributes = poison;
}).get
"n"
"val";
};
};

flake.tests."recorded-deps" = {
test-recordedDeps-declared = {
expr = genScope.recordedDeps { declaredEdges = id: if id == "b" then [ "a" ] else [ ]; } "b";
expected = [ "a" ];
};
test-recordedDeps-empty = {
expr = genScope.recordedDeps { declaredEdges = _: [ ]; } "x";
expected = [ ];
};
};
}
Loading
Loading