feat: pipe.broadcast cross-scope push + collect reads exposed#623
Merged
Conversation
Add `pipe.broadcast pred` — a push routing primitive, the dual of
`pipe.expose`. A scope distributes its source-transformed pipe value to
every other scope whose context matches the receiver predicate,
fleet-wide; receivers read the pipe normally. Receiver-only predicate
(same signature as `collectAll`), self-excluded. Implemented as a
`collectAllBroadcast` pass (1b) mirroring `collectAllExposed`, reusing
`findMatchingAll` entity-kind filtering and `resolveThunks` cross-host
config resolution. `bindsPipeLocally` gains a broadcast clause so a pure
receiver does not fall through to ancestor inheritance.
Also make `collect`/`collectAll` read raw + exposed values at each
source scope rather than raw emits alone, so a peer's collect sees data
that children `pipe.expose`d up into a host. Fixes the
expose-then-fleet-collect path (witness test-expose-then-fleet-collect).
Tests: new pipe-broadcast suite (all-to-all, user->remote-host,
source-transform, predicate scoping incl. { host, user } compound,
self-exclusion) and pipe-broadcast-isolation suite (cross-pipe-name,
host-target-excludes-home entity-kind isolation, no-match predicate, and
the broadcast<->collect boundary: broadcast-injected values are not
re-collected). Full CI 1036/1036.
Resolve pipe config-thunks against the PRODUCING class module + scope, not the consuming one. The prior behavior was a latent bug: a host- produced thunk consumed in a home resolved against the home config. - Cross-scope (broadcast/collect): resolveEntry resolves a source's config-thunk against the producer's class config — host -> nixos, user/home -> home-manager (hostConfigs.<host>.home-manager.users.<name>). - Deferred (__configThunk): the marker carries the producer's class + name; the class-module wrapper resolves it against the producing class's config, handing each thunk both `config` (producer class) and `osConfig` (the enclosing host), mirroring home-manager. osConfig is requested only when a marker needs the host config, so standalone homes keep working. - isConfigDependent now also detects osConfig-only thunks. Review fixes: pin the bindsPipeLocally broadcast clause with a pure- receiver test; extract a shared dedupEffectsByPolicy helper; document the host-broadcaster raw-only boundary. Tests: new pipe-config-scope suite (host->home, user->own-home, user->host) plus broadcast config-thunk cases; the exposed-config-thunk witness now reads osConfig. Full CI 1041/1041.
The producer-class config-thunk resolution hard-coded the home-manager battery's delivery convention into core fx (the `home-manager.users.<name>` path and the `"homeManager"` class literal). Replace that with a registry-driven route. - New `den.classes.<class>.hostPath` (name -> path): where a class's members nest inside the enclosing host config. The home-manager battery registers it from the same value as its forward delivery path (single source of truth); host-level classes (nixos, darwin) leave it null. - assemble-pipes (producerConfigs / producerOf) and class-module (resolveMarkers) now navigate host<->producer config via the producer's registered hostPath and decide "is this class host-nested" from the registry, keyed by scopeEntityClass — no `home-manager`/`homeManager` literals in core fx. A new home-style class works by registering hostPath, no core edits. - Marker carries `__producerClass` (was `__producerKind`); scopeEntityClass threaded into assemblePipes alongside scopeEntityKind. The osConfig handed to deferred thunks remains den's own host-config link (home.nix extraSpecialArgs.osConfig, derived from the host's intoAttr). Behavior unchanged; full CI 1042/1042.
…g literals)
Generalize the producer-class config-thunk machinery so it no longer assumes
NixOS — den can describe a config-owner that isn't a host (e.g. terranix,
nixidy) without core edits.
- `den.classes.<class>.hostPath` -> `parentPath` (name -> path within the
enclosing config-owner), and a new `parentArg` (the module arg by which a
nested member reaches the owner config; home-manager registers "osConfig").
Both null for root classes that own a top-level config.
- The home-manager battery registers parentPath + parentArg = "osConfig".
- assemble-pipes: `isConfigDependent` detects `config` or ANY registered
parentArg (not the `osConfig` literal); `producerConfigs` returns
{ config, owner, parentArg } and navigates via the producer class's
parentPath; resolveEntry hands a nested producer's thunk the owner config
under its class's parentArg. Naming reframed host -> enclosing config-owner.
- class-module: the wrapper reads each class's parentPath/parentArg from the
registry — a nested consumer fetches the owner via its own parentArg, and a
thunk is handed the owner under its producer's parentArg. No osConfig literal.
`hostConfigs` was already generic over instantiates (any entity with an
intoAttr output), so the model already supported non-host owners; this removes
the remaining NixOS-flavored literals from the new config-thunk path. Behavior
unchanged; full CI 1042/1042.
68c8531 to
a55556b
Compare
drupol
approved these changes
Jun 26, 2026
drupol
left a comment
Collaborator
There was a problem hiding this comment.
Not on a computer at the moment, can't read the code efficiently. That said, the feature looks nice!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a cross-scope broadcast primitive to the pipe/quirk system, makes
collectcompose withexpose, and fixes config-thunk resolution to honour the producing class + scope — then makes that resolution class-neutral so it no longer assumes NixOS.pipe.broadcast pred— push primitive (dual ofpipe.expose)Where
pipe.exposeroutes a value up to the parent applying source-side stages,pipe.broadcastroutes the (transformed) value laterally to every other scope matching a receiver predicate, fleet-wide. Receivers read the pipe normally — nocollectpolicy required.collectAll); selects receivers by their own context — kind, name, tags,host. Compound predicates like{ host, user, ... }: host.name == "edge"target user scopes on a specific host.S≠R) — a source represents itself via its own base.collectAllBroadcast(Pass 1b) mirroringcollectAllExposed; reusesfindMatchingAllentity-kind filtering.bindsPipeLocallygains a broadcast clause so a pure receiver doesn't fall through to ancestor inheritance.collect/collectAllread raw + exposedcollectTaggednow readsresolveThunks(raw) ++ allExposed.${sid}, so a peer's collect sees data that childrenpipe.exposed up into a host. Fixes the expose-then-fleet-collect path.allExposedis Pass-1 data → no eval cycle.Producer-class config-thunk resolution
A pipe config-thunk now resolves against the producing class module + scope, not the consuming one (the prior behaviour was a latent bug — a host-produced thunk consumed in a home resolved against the home config).
__configThunk): the marker carries the producer's class + name; the class-module wrapper resolves against the producing class's config and hands each thunk bothconfig(producer class) and the owner config (the enclosing host), mirroring home-manager. The owner arg is requested from the module system only when a marker needs it, so standalone homes keep working.isConfigDependentalso detects owner-config-only thunks.Class-neutral resolution (no host/osConfig hard-coding)
The host↔producer navigation is registry-driven, not baked into core fx — so den can describe a config-owner that isn't a NixOS host (e.g. terranix, nixidy) without core edits:
den.classes.<class>.parentPath—name → pathlocating a member within the enclosing config-owner (the same route its content is delivered to);den.classes.<class>.parentArg— the module arg a nested member reaches the owner by. Bothnullfor root classes that own a top-level config (nixos, darwin, …). The home-manager battery registersparentPath+parentArg = "osConfig"from the same value as its forward-delivery path.producerConfigs/resolveEntry(assemble-pipes) and the class-module wrapper navigate via the registry, keyed byscopeEntityClass.isConfigDependentdetectsconfigor any registeredparentArg. Zerohost/osConfig/home-managerliterals remain in core fx.hostConfigswas already generic over instantiates (any entity with anintoAttroutput), so the model already supported non-host owners; this removes the remaining NixOS-flavored literals. (Building actual terranix/nixidy entity/class defs is follow-on work this unblocks; the two softor "nixos"class-default fallbacks are left intact.)Tests
pipe-broadcast(10): all-to-all · user→remote-host · source-transform · predicate scoping (incl.{ host, user }compound) · self-exclusion · targeted no-leak · pure-receiver binding · config-thunk from host / user source.pipe-broadcast-isolation(4): cross-pipe-name · host-target-excludes-home (entity-kind via sharedhostcontext) · no-match predicate · broadcast↔collect boundary (a fleetcollectAllcounts each raw emit once, not the broadcast-amplified view).pipe-config-scope(3): host→home, user→own-home, user→host producer-class resolution.pipe-scope: thetest-expose-then-fleet-collectwitness now passes; theexposed-config-thunkwitness reads the owner arg.Full CI: 1042/1042.
treefmtclean.Commits
pipe.broadcastcross-scope push + collect reads exposed