On-network reduction as the default driver + browser visualization of propagator-network execution#64
Open
kumavis wants to merge 67 commits into
Open
On-network reduction as the default driver + browser visualization of propagator-network execution#64kumavis wants to merge 67 commits into
kumavis wants to merge 67 commits into
Conversation
…ropagator network with execution playback for arbitrary programs; grounding audit (5 facets + critic): PTF Track 1 already provides capture (bsp-round/cell-diff/topology JSON WITH edges + observatory) and the VS Code viewer — the gaps are the headless CLI exporter, the standalone browser host, and the Tier-1 observer dropout; HANDOFF reconstructed post-halt (was 33 iterations stale); environment-migration posture ledgered OWNER-PROVISIONAL https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…rens PATH fallback — container shakeout found the 9.0 pin is real (thread #:pool 'own at propagator.rkt:3748 rejects apt 8.10); Racket 9.0 installed; probe run pending the 9.0 recompile https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…agator edges EXIST at HEAD (March '0 edges' empirically stale); headless capture works end-to-end (17 rounds/35 diffs/152 fires/31.5KB JSON); subsystem categorization hollow post-universe-migration (F4 -> Phase 1 design input); LSP trace-capture path flagged structurally dead (F7); probe requires errors.rkt (first-run fix); ledger + dailies + HANDOFF for iteration 46 (Stage-3 design lock) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…er envelope w/ timestamp round-correlation; schema reuse + exporter-side identity supplement (named scaffolding, fold-in point pre-registered); dependency-free single-file viewer; domain-based identity stack; G3 defer-with-visibility; NTT-N/A named — two independent adversarial critics running before lock https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ependent critics: 2 BLOCKERs + 8 MAJORs adjudicated; tools-only Phase 2 OVERTURNED (Tier-1 observer fix promoted to 2b — fire-once programs traced EMPTY without it); PATH B decided (identity exporter-local, fold-in dangle killed); D4 downgraded per in-round measurement (probe extended: 55% cell-domains coverage, 7 domains); D7 semantic value detail added; Phase 0A acceptance file added per critique; VAG challenged + overturned two inherited patterns https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…s + relations/solve + NAF coverage (viz concerns SS1-SS4); Level-3 probe validation running, corpus audit results to follow https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…mmands/0 errors, after the gate caught syntax-doc drift: int+/int* map partials fail inference at HEAD, plus-forms work — owner flag); corpus audited (max 87c/17p, far under the 1k layout trigger); A4 free-path PARTIALLY VALIDATED positive (round snapshots carry solver topology; BSP-observable solve share = the 2c question); probe prints per-command errors; next: 2a full-suite baseline https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…e GREEN: 8658/439 ALL PASS (401.5s); 9 initial failures all adjudicated environment-setup (pkg link x6, rackcheck x2, stale 8.10 .zo x1), fixed + individually re-verified + one forced regression gate; container re-setup recipe recorded; 2b (Tier-1 observer edit) unblocked https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…(gap G3) — armed path fires with per-fire eq?-pruned champ-diff for precise cell-diff attribution + emits ONE bsp-round mirroring Tier-2; unarmed fold byte-identical (zero cost); 3 unit tests pin armed/attribution/unarmed; corpus note: none of the 3 corpus files take Tier-1 (falsified-workload data point — the fix covers fire-once workloads, proven by direct unit test) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…, 400.6s ~= baseline); perf adjudicated structurally + clean A/A distribution captured (container noise floor: same-code 15.3% 'significant' — calibration bar recorded); falsified-workload note (corpus never takes Tier-1); bench-ab --ref doc-drift flagged; next: 2c exporter https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…he vizTrace/1 envelope: per-command captures w/ topology, epoch-bucketed timestamped rounds, per-epoch last-snapshot topologies (the A4 solver free path), per-topology identity (D4 measured coverage; cell-id spaces are per-network — impl note), bounded D7 value detail, always-computed validation block; ALL 2c criteria PASS on the 3-file corpus (monotone, captures==commands, solver epochs visible: relation epochs 67-81c/172-179p vs 41c/0p elab); golden test 7/7 https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…DOFF for iteration 51 (Phase 3: the standalone viewer) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…/index.html, single-file, dependency-free (locked D3): component-aware layered Canvas layout w/ self-loop arcs, zoom/pan/hover, D4 identity coloring with measured-coverage display, epoch selector + round scrubber playback (fired propagators highlight, diffed cells flash, old->new diff list); tools/viz/check.js = headless node verification of the pure core — ALL CHECKS PASS on the 3-file corpus (scopes 32/33/8, max 260 nodes, all positioned, bucketing complete); browser acceptance = owner's machine (no GUI here, recorded) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
… -> iteration 52 = TRACK CLOSE (riders adjudication, PIR + autonomy retro, Master row, pause adjudication) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…harter SS9 autonomy retro), riders adjudicated (2 dissolved/1 dormant/2 deferred), Master PTF Track V row (naming collision disambiguated), DEFERRED riders + owner flags, dailies interval closed, HANDOFF rewritten to the PAUSED state with the owner review queue + re-arm instructions https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…network, not a cloud of dots Owner feedback: expected labeled propagators firing cells->cells + scheduler data + one network that grows as execution happens; saw mostly-unconnected nodes and opaque scope/epoch jargon. Diagnosis (grounded): only 5-22% of nodes were connected — the rest are unwired infrastructure cells (registries, fuel, relation store) the compiler reads directly, not via propagators; I was dumping the whole network and defaulting to the 0-propagator finalTopology. - exporter: add source-line map (propagators label by their source construct; fire-fns are anonymous closures, srcloc is the identity) + per-epoch firstRound for temporal ordering. Golden test 7/7. - viewer: unified timeline (grow + fire + flow over one scrubber/play); default filter to the connected propagator subgraph (hide unwired, keep named cells that change e.g. the relation store); scheduler panel + growth sparkline; source-construct propagator labels; killed scope/epoch jargon. - check.js: rebuilt for the new model. Default-view connectivity 5-22% -> 100% on all 3 corpus traces; acceptance propagator labels 179/179 from source. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ed layout + mp4 render Owner feedback drove three changes: 1. CORRECTNESS: render each round against its OWN network snapshot, not a per-epoch one. A round's fired props / cell-diffs live in the id-space of the network THAT ROUND ran on (elaboration vs a solve-fork); the old per-epoch keying conflated networks, so fired propagators showed detached from their cells. Exporter now emits a deduped per-round topology table; each round carries a topo index. Golden test pins the soundness invariant (fired props + diff cells are in the round's own topology). 2. LAYOUT: force-directed (Fruchterman-Reingold, deterministic seed) added alongside layered, switchable in the UI. Fits the actual shape (self-loop refiner swarms around shared cells) better than layering, which stretched the low-branching graph into a vertical stack. 3. EXECUTION FOCUS: aimed at the relational/solver demos (where program execution lives on the network); propagators labeled by source construct; unwired infrastructure cells hidden by default. Measured finding (recorded): execution is near-sequential at the round level (max ~2 propagators fired per round across propagator-demo / sudoku / relational); the concurrency Prologos has today is branch-level worldview/NAF FORKING, which shows as the network changing across the timeline. Wide intra-round concurrency awaits constraint-propagation / on-network reduction. render-mp4.js renders the timeline to mp4 (SVG->rsvg->ffmpeg). check.js + golden test updated to vizTrace 2; ALL CHECKS PASS on the corpus. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
… (corrects an earlier wrong claim)
Owner challenged my claim that functional reduction isn't on the network. They
were right: the PReduce series put reduction-as-rewriting on the propagator
network (eclass-graph.rkt installs union + congruence propagators), gated behind
current-preduce-ingest? (env PREDUCE_INGEST; default off, 'NOT a deployment
path'). My exports ran without it, so they captured zero reduction.
- viz-export.rkt: #:reduce? kwarg + --reduce flag parameterizes
current-preduce-ingest? around process-file, so β/δ/ι reduction runs as
e-graph propagators and the BSP observer captures it.
- Two findings that correct earlier statements in this thread:
(a) concurrency is NOT inherently ~2/round — that was tiny toy workloads; a
balanced arithmetic tree reduces 256 propagators wide in one round.
(b) on-network reduction is real (just gated). WITHOUT --reduce the parallel
tree folds in 1 round (off-network recursive reducer); WITH --reduce it's
240 rounds, 256-wide — reduction IS the propagator network when enabled.
- examples/2026-06-14-{reduction-showcase,parallel-reduction}.prologos: clean
(0-error) reduction workloads; parallel-reduction is the wide-concurrency demo.
- check.js: layout once per distinct topology (was per-round → pathological on
the 300+-round reduction traces).
https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
Naive doubly-recursive fib. With --reduce (on-network reduction), the e-graph's hashcons shares every structurally-identical [fib k], so the exponential recursion collapses to a LINEAR DAG: naive fib(15) = 1973 recursive calls, but the propagator network has only ~18 propagators / ~90 cells. Demonstrates that a non-memoized program is automatically memoized by structural sharing on the e-graph. (Visual caveat: e-class nodes render as opaque hash() cells; the structural collapse is the story, not per-node values.) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…) nodes) serialize-lattice-value now unpacks e-class product cells (eclass-cell.rkt): the :best component is a (cost . reduced-form) pair, so render the form via pp-expr. Reduction cells now read as their actual values (e.g. '(lit 14)') instead of 'hash(5 entries)'. Viewer labels nodes by that value (cleaning '(lit N)' -> 'N'), so reduction DAGs show numbers, not cN. (Value-rendering improvement; distinct from the D2 identity-envelope freeze.) https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
Owner: 'how do the disconnected components relate?' Answer (grounded): they relate through the hashcons registry (content-addressing), but that's accessed by the intern FUNCTION imperatively, so it's never in the topology — and the redex structure is consumed (an e-class's :best becomes the reduced VALUE). Fix: capture parent->child structure at INTERN TIME (before it's consumed), gated behind current-eclass-containment-box (default #f, off-network viz side-channel, zero production cost — eclass-graph.rkt). The exporter sets it under --reduce, resolves children via the registry, and emits per-topology containment edges (filtered to present cells). Viewer + mp4 renderer draw them DASHED VIOLET, distinct from solid propagator dataflow. Result on fib(15): components 77 -> 3 — the scattered equality-pairs become one connected computation DAG (610<-377,233 ; 377<-233,144 ; shared via the result e-classes). Gated change; targeted eclass + viz tests green (84). https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
Rewrites the viewer from vanilla-Canvas to React + SVG with a live d3-force simulation (deps from CDN: react, react-dom, d3, @babel/standalone). Force mode now shows the graph SMOOTHLY settling — stepping the timeline reuses node positions and gently restarts the sim (alpha) so new nodes ease in and the graph relaxes, instead of the previous jerky static-layout snap. Layered mode stays a static layout. All features preserved: value labels, dashed-violet containment edges, fired/born/written highlights, scheduler sparkline, infra toggle, hover, pan/zoom, fit. The @pure data core is unchanged (check.js + mp4 renderer still use it); JSX verified to compile; full suite 8674 green. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…peration - Speed slider in the sidebar controls playback step interval (80–1600ms). - Propagators whose inputs == outputs (the e-graph union-emitter) now label 'union ≡' instead of inheriting the generic source line ([fib 15]). This answers 'what does this propagator do' — it's the e-graph union (structural merge of two e-classes), not anything fib-specific. JSX compiles; check.js green. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
… crash - 'collapse e-classes' toggle: unions every pair of cells joined by a union ≡ propagator into one e-class node and drops the unions, so each value is a single node and the containment DAG reads cleanly (fib round 13: 19 nodes → 8, doubled cells merged, values 11/12/13/14 as single nodes). Collapsed nodes carry members + mergedProps so fired/written/born highlights still map back (a union firing lights its e-class node). - FIX 'TypeError: Cannot read properties of null (reading x)' on interaction: d3-force was dereferencing a link whose endpoint wasn't a node in the sim. Links are now filtered to endpoints present in the simulation's node set. JSX compiles; check.js green. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ater 'TypeError: reading x of null' on background drag: onMove passed a setView updater that read dragRef.current.x when it ran; if the drag ended first (onUp set dragRef.current=null) the updater dereferenced null. Capture the drag offset before the updater; guard wrapRef too. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…tive written down Owner: this is a prototype branch for visualizing a FUTURE propagator-native Prologos, built on the PReduce on-network-reduction prototype; on this branch we ALWAYS use that reduction and the viz shows it. So: - viz-export.rkt: --reduce becomes the default (#:reduce? #t); --no-reduce opts out. - design doc: § BRANCH DIRECTIVE written down, with the honest precise reach of 'propagator-based reduction' today (redex⇒result rewrites ARE union propagators = DPO rewriting, PRN §2; RHS compute + recursion driver are not-yet on-network, Track 8 future). Ledger entry recorded. Golden test green. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…on as propagation' Stock-take after the goal clarified: what we have (capture + React viewer + on- network reduction default), the 3-layer gap (legibility / coverage / generality), and a phased plan — Phase A (make on-network reduction legible: rewrite labels, reduction-step readout, directed containment), Phase B (complementary reduction- trace lens), Phase C (PReduce endgame: compute + recursion propagator-native). Key property: the viewer renders whatever propagators a trace contains, so Phase C lands with no viewer changes. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…wner caught it) reduction.rkt is the off-network recursive reducer; instrumenting it would visualize the off-network reduction, against the branch directive. Removed the 'reduction trace' bridge. Refined the gap by program shape (verified on the fib trace): arithmetic-shaped computation (parallel-reduction) is already computation-as-propagation on the network; recursion-shaped (fib) has its essence off-network until Phase C moves recursion + compute on-network. Plan is now A → C. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
… / this-round / edges The legend conflated cell fill (identity) with ring color (this-round state) and didn't explain shapes. Reorganized into labeled sections: shapes (● cell / ◆ propagator); cell fill = identity (well-known infra gold, type-inference green, multiplicity purple, infra gray, + the lattice domains actually in view); this round (red ring = fired/merged, gold fill = written, teal ring = new); edges (solid = dataflow, dashed violet = containment). Rings vs fills now drawn as rings vs fills to match the canvas. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
A [fib 6] companion to fib-naive ([fib 15], which hits the Phase 4a perf wall on export). Exports in 172 rounds with up to 18 propagators firing per round — the recursion shown AS union propagators (the Phase 4a viz goal). Comment documents the no-memo-collapse finding and the SH/Zig perf frontier. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…s naive recursion Key the β redex by the NORMALIZED arg (whnf'd), so recursive calls reaching the same value share ONE e-class and the recursion MEMO-COLLAPSES. Reduction stays CALL-BY-NAME (#:compute substitutes the ORIGINAL arg); only the memo KEY is normalized. Sound: arg-nf and arg denote the same value ⇒ identical contractum (whnf deterministic). Key-normalization is BOUNDED (private fuel box — never depletes the real budget) and GUARDED (a divergent/unused arg falls back to the original redex key: no collapse, no regression, no spurious divergence). Effect: naive fib goes exponential→LINEAR. reduce_steps fib 8/12/15/20 = 202/287/350/455 (was 741/exponential/>120s). fib 15 reduce 274ms (was >120s); fib 15 viz export 16s/234 rounds/0 errors (was timeout). fib 10 = 55 verified. Answers fib-naive.prologos's own question: WITH a cbv memo key, the hashcons now SHARES [fib k] subterms (collapses the tree). test-preduce-ingest 33 green. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…nature Drop the per-round O(cells + every cell's value) cost in viz-export's intern-topology!. The dedup signature is now computed directly from the pnet (cell-ids + propagator connections only) via topology-signature, skipping the per-cell serialize-lattice-value that dominates serialize-network-topology (e-class values carry :best/:alts/:canonical/:provenance sets). The full serialize now runs ONLY on a NEW topology, not once per round — under monotone network growth most rounds reuse a topology. Signature matches the legacy format exactly (cell ids sorted | pid:in>out sorted), so dedup is unchanged. fib 15 export 16.3s→11.1s (~32%), identical output (234 rounds, 120 topologies, 0 errors); fib 6 trace verified well-formed (48 topologies, all referenced, 0 invalid indices). Also adds cell-22 (dispatch-request) to the well-known-cells identity table (missed in Phase 2). https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…v key + incremental observer) cbv memo key (naive fib exponential→linear; fib 15 >120s→0.27s reduce, traceable) + incremental viz observer (fib 15 export 16.3s→11.1s). Suite 8673 all-pass, faster (372s). Updates the Phase 4 design doc §9, plan tracker, and ledger. The DRIVER (whnf) remains off-network — 4b/4c frontier; Phase 5 owner-gated. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ion via demand cascade + extraction bypass Stage-3 design for moving the reduction DRIVER on-network (4a moved the recording). Core primitive: whnf-step1 (one-step classifier: contractum | demand | whnf | native) derived from whnf-impl/match; a reduce-propagator + demand-latch cascade on the dispatch stratum drives reduction; whnf/nf become intern→saturate→extract (Phase 5 bypass). Form-rep decision: expr-structs + native one-step compute (sidesteps the list-form template's de-Bruijn-subst gap). PARITY gate (egraph-whnf == native-whnf) per migrated construct is the non-negotiable safety net. Honest staging: whnf-impl/match is ~1700 lines over dozens of constructs → complete bypass is multi-session; native fallback shrinks as constructs migrate; terminal (delete native) owner-gated. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…f-via-egraph (parity-gated) The foundational primitive for network-driven reduction (design §3): whnf-step1 classifies a term into 'whnf | 'native | (step C) | (demand SUB RECON), mirroring whnf-impl/match arm-for-arm for the migrated fragment (β, ι natrec, suc-collapse, fst/snd, J, boolrec, ann, vhead/vtail, + subterm-demand arms). whnf-via-egraph is the intern→reduce→extract driver (the shape Phase 5 generalizes to scheduler-driven saturation): iterate whnf-step1, satisfy demand via nested whnf-via-egraph, detect neutral-stuck via no-progress. PARITY GATE (test-preduce-egraph.rkt, 27 checks): whnf-via-egraph == native whnf across a corpus (β chains, ι, suc, projections, boolrec, J, ann, higher-order, arith-via-native, neutral/stuck, values), and the migrated arms are confirmed as step/demand (not silent native). Pure addition — default whnf/nf untouched (opt-in); the 'native fallback makes any non-migrated construct parity-trivial. The scheduler-driven cascade (reduce stratum: keep-pending + worklist re-trigger) is the staged next increment — this validates the one-step decomposition is parity-correct and de-risks it. Honest staging per design §7. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ec (§9) Records 5a (whnf-step1 + whnf-via-egraph, parity-gated, suite 8705) in the design doc, plan tracker, and ledger. Adds §9: the complete, de-risked 5b machinery spec (origin-keyed reduce-request keep-pending stratum; the step-arm union re-triggers the cascade each round; emitter kicks the first pass; K :best-write extraction; 6-point risk register) so 5b implements cleanly next. 5a is the landed increment (validated one-step primitive; default whnf untouched); 5b/5c (scheduler cascade + bypass deploy) are designed, parity-gated, terminal owner-gated. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…genuine network-DRIVE)
The reduction DRIVER is now the BSP scheduler, not a Racket loop (design §9). A
keep-pending reduce stratum drives the head cascade; whnf-via-egraph-network
interns the term, kicks an emitter, run-to-quiescence saturates, and the WHNF is
EXTRACTED from the origin class :best.
- propagator.rkt: cell-23 reduce-request (origin-keyed {K → current-form};
hash-overwrite; provide + preallocate net13 + drift check).
- reduction.rkt: process-reduce-requests (register-stratum-handler!
#:keep-pending? #t #:tier 'topology) — 'whnf/'native write K's cost-0 :best
(extraction); 'step interns+unions K (the union is the worklist activity that
RE-TRIGGERS the stratum each round — the load-bearing cascade mechanism) +
re-requests {K → C}; 'demand reduces the strict subterm natively then continues
the cascade. whnf-via-egraph-network driver (emitter + quiescence + extract;
no-plumbing/inadmissible → native).
- viz-export: cell-23 in the well-known-cells identity table.
Network Reality Check PASSES: scheduler-driven stratum + emitter + per-step union;
result via net-cell-write; trace intern→emitter→reduce-request→stratum→cascade→
extract. PARITY GATE (test-preduce-egraph.rkt network variant, +20 checks, 47
total): whnf-via-egraph-network == native whnf across the corpus. cell-count test
bumps for cell-23 (propagator/trace-serialize/observatory). Demand subterms
reduced natively here (full cascade-driven demand via set-latch = future);
the HEAD chain is scheduler-driven.
https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ed (suite 8725) 5b ✅ — whnf-via-egraph-network (genuine network-DRIVE; Network Reality Check passes; parity-gated; suite 8725). Updates the design doc (§10), plan tracker, and ledger. 5c (route default whnf through the engine = bypass deploy) staged — low-value + hot-path-risky until more constructs migrate off 'native; terminal owner-gated. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
… 5b scheduler cascade Factor the vizTrace-2 envelope builder (build-viz-envelope) out of viz-export-file (behavior identical — fib 15 still 234 rounds/120 topologies), and add viz-export-expr-network: drive a constructed ground expr through whnf-via-egraph-network (the Phase 5b scheduler-driven reduce-stratum cascade) with the observer armed, so the trace SHOWS the network DRIVING reduction (the reduce-request cell-23, the emitter, the per-step union cascade) — not just recording it. Sets up the e-graph plumbing inline. Demo cascade (12 nested (λx.x) over (suc 4)): 43 rounds, 15 topologies, 0 errors, cell-23=reduce-request active, up to 13 propagators/round. viz-only (not in the suite path); default viz-export-file unchanged. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…whnf-step1 Toward deleting off-network reduction: migrate int +/−/*/neg/abs/lt/le/eq into whnf-step1 (the one-step classifier) — fold when both operands are int literals, else demand the non-value operand (strict). Mixed/rat/posit (both values, not int) and div/mod (zero-guard) stay 'native — parity-safe. Shrinks the 'native fallback (int arith is now scheduler-cascade-drivable). PARITY gate (test-preduce-egraph.rkt, 72 checks): whnf-via-egraph / -network == native whnf across folds, nested strict-demand ((1+2)*(10-3), (2*3)+(4*5)), comparisons, β-exposing-arith ((λx.(x+1)*(x-1)) 5 → 24), div/mod-native. Pure addition — default whnf untouched. Remaining batches: structural reduce/match, δ, rat/posit, from-nat, foreign, maps, strings; then route the default + delete the native arms (the multi-iteration grind; perf-feasibility checkpoint at routing). https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…, δ, meta) Toward deleting off-network reduction. The full inventory is ~250 whnf-impl/match arms, so per-arm hand-corpus testing is impractical — pivot to routing the default whnf through the cascade and using the full suite as the comprehensive parity oracle + perf checkpoint. - Routing hook: current-egraph-whnf? (default OFF; PREDUCE_ROUTE=1 flips it on). whnf dispatches to whnf-via-egraph-network when on, whnf-core (the unchanged native body) when off. The cascade's native fallbacks use whnf-native (a parameterize ([current-egraph-whnf? #f]) wrapper) so native one-step compute + its internal recursion never re-enter the cascade — no infinite recursion. Production DEFAULT is unchanged (opt-in). - Batch 2: migrate structural reduce/match, δ (fvar global-def unfold), and solved meta into whnf-step1, reusing the native compute helpers (try-structural-reduce, try-builtin-reduce, global-env-lookup-value) — moving the DRIVING on-network, not reimplementing the compute. Acceptance file routed (PREDUCE_ROUTE=1): 0 errors, reduce_ms 247→297 (~20% overhead — routing the default through the cascade is correct AND fast for the fib/factorial/arith/reduce/δ fragment). Parity test 72 green. Full suite routed is running (the suite-wide parity oracle) — un-migrated arms ride the native fallback; remaining batches (rat/posit/quire/generic arith, maps/pvec/set, FFI, …) empty out 'native, then delete the native arms. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ff (it tests the 4a path routing replaces) The routed suite (PREDUCE_ROUTE=1, default whnf through the cascade) was 8749/8750 green — the ONLY failure was test-preduce-ingest, and all 4 sub-failures are about the e-graph RECORDING STRUCTURE (class persistence / counts from preduce-ingest-delta), NOT reduction-result parity (every reduction VALUE matches). That file specifically tests the 4a recording path that routing bypasses (the cascade records via eclass-union, covered by test-preduce-egraph). Pin it routing-off until the native reducer is deleted, at which point its recording assertions retire. Routed suite is now a clean oracle. MILESTONE: routing the default whnf through the scheduler cascade is suite-clean (correct reduction across all 441 files), ~20% overhead. Validates the deploy path; remaining = migrate the rest of the ~250 arms to empty 'native, flip the default, delete the native reducer. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…tion driver
Owner directive (2026-06-15): 'delete the recursive off-network DRIVER, keep the
compute leaf.' Flip current-egraph-whnf? default → on: whnf routes through the
scheduler-driven cascade (whnf-via-egraph-network) by default. The BSP scheduler
now DRIVES ground reduction (β/ι/δ/app/reduce/proj/… via whnf-step1 + the reduce
stratum); the recursive native reducer is no longer the default path.
The native reducer (whnf-core/whnf-impl) is RETAINED but DEMOTED from driver to two
leaf roles — NOT 'off-network reduction':
(a) the primitive COMPUTE LEAF reached via whnf-step1's 'native (arithmetic
folds, data-structure ops, FFI — 'compute inside the rule', the e-graph's
own design);
(b) the NON-GROUND fallback — the e-graph requires PCE-admissible (ground)
terms, so metavar/elaboration reduction can't be interned and de-routes via
whnf-native. A fundamental admissibility limit, not a gap.
PREDUCE_NATIVE=1 forces the native driver (e.g. test-preduce-ingest's 4a-recording
test). GATE: full suite with the cascade as default = 8750 ALL PASS (386.9s,
~15% over native 335s — acceptable per owner).
https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…estone + the two architectural limits The cascade is the default reduction driver (suite 8750 all-pass, ~15% over native). Records the owner-calibrated scope (delete driver, keep compute leaf) and the two fundamental limits: non-ground reduction can't be PCE-interned (native fallback retained), and full whnf-impl deletion is a reimplementation (higher-order recurse + FFI), not a mechanical grind. Updates ledger + plan tracker. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…nds on-network
Suggested refinement: when whnf-step1 hits 'native (a primitive compute construct —
arith fold / data-structure op / FFI not yet migrated), the cascade now reduces it
via whnf-impl directly (ambient routing ON) rather than whnf-native (de-routed). So
the construct's OPERAND sub-reductions route BACK through the cascade (on-network);
only the primitive fold itself is native ('compute inside the rule'). NOT whnf F
(would re-route F → infinite loop); NOT whnf-native (de-routes operands too).
Demand subterms stay native: routing them too is correct but ~7x slower (every
recursive subterm nest-cascades); the head chain is already on-network via the
'step cascade. GATE: full suite 8750 all-pass (294.6s — faster than the prior
default-routed 386.9s, ~88% of native).
https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…declined + architecture end-state #1 (route 'native compute-leaf operands on-network) done + validated (0c51010, suite 8750, faster). #2 (migrate folds) declined — it's in tension with #1 (trades operand-on-network for fold-viz, or ~7x for both). Records the Phase 5c end-state: cascade drives ground reduction by default; native is the compute leaf + non-ground fallback. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…arkline Owner feedback: the [fib 6]/defn-fib propagator labels were source LOCATIONS (where installed), not what the propagator does — misleading. Now: request emitters (write a *-request stratum-trigger cell) → 'emit reduce'/'emit dispatch'; union relates → 'union ≡' (already); others keep the source label. Also shrink the scheduler sparkline (H 46→24 — it's a thin trend line). Viewer-only; relabels existing traces. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
Real cause (owner): the global 'svg { width:100%; height:100% }' CSS rule
overrode the sparkline's height={H} attribute, so it stretched to fill its
container. Fix: give the scheduler SVG an inline style (width/height/flex:none)
— inline style wins over the stylesheet — plus a viewBox so the path scales.
Reverts the earlier H 46->24 tweak (that addressed the symptom, not the cause).
https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
…ds reduce in the same BSP rounds Single-subterm 'demand was strict in exactly one position, so binary arithmetic (int+/-/*/lt/le/eq — strict in BOTH independent operands) sequenced operand i fully before operand j began. For (int+ (fib (n-1)) (fib (n-2))) the two recursive branches reduced one-then-the-other, violating the mantra's "all in parallel". whnf-step1's binary-arithmetic arms now emit 'demand-par via the step1-par helper. The reduce stratum's pr-demand-par interns every non-value operand to its own e-class and queues them into the SAME reduce-request round, so their cascades interleave per BSP round; a barrier propagator (set-latch fan-in, per propagator-design.md) joins them — when all operand classes reach a cost-0 :best it re-forms the head and the head cascade continues. Value operands pass through unchanged (no class allocated). Inadmissible operand -> native fallback (parity). Network Reality Check: the operand reductions are now genuine on-network cascades (net-add-propagator / net-cell-write), vs whnf-native off-network in 5b; the barrier is a propagator; the join is information flow through the operand cells. Evidence (viz before/after): the balanced x/+ tree collapsed 353 -> 43 BSP rounds (8.2x critical-path depth reduction, max 256 propagators in one round); fib 6 gained concurrency 51 -> 78 multi-propagator rounds. fib + tree results identical on-network vs native (8 and 1462). The iterating whnf-via-egraph variant handles 'demand-par by reducing operands sequentially (parity-identical; the parallelism is a property of the network driver, not the classifier). Parity-gated (test-preduce-egraph.rkt both variants, 72 checks) + 250 targeted tests green. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
Adds a 'labels' checkbox in the header (default on) that gates the on-canvas node text. Lets you declutter dense networks while keeping the hover tooltip and legend for identification. https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
kumavis
changed the title
There was a problem hiding this comment.
Pull request overview
This PR turns scheduler-driven, on-network reduction into the default WHNF driver (keeping native reduction as a compute-leaf / non-ground fallback) and adds a browser + headless tooling stack to visualize propagator-network execution from exported vizTrace2 envelopes.
Changes:
- Add
vizTrace2export tooling (viz-export.rkt) plus headless verification/render helpers (tools/viz/check.js,render-mp4.js) and golden tests. - Implement/route WHNF through the reduce-request stratum (
whnf-step1,whnf-via-egraph-network, parallel operand demand via barrier), and update dispatch to be stratum-driven. - Update core infrastructure + tests/docs/examples to reflect new well-known cells (dispatch/reduce request) and new reduction driver behavior.
Reviewed changes
Copilot reviewed 39 out of 39 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/check-parens.sh | Add PATH fallback for locating racket when macOS-pinned binary isn’t present. |
| racket/prologos/trace-serialize.rkt | Improve lattice-value rendering for e-class cells by surfacing :best reduced forms. |
| racket/prologos/tools/viz/render-mp4.js | Headless SVG frame generator for vizTrace2 timelines (for mp4 pipelines). |
| racket/prologos/tools/viz/check.js | Headless invariants checker for vizTrace2 + viewer “pure core”. |
| racket/prologos/tools/viz-export.rkt | Exporter producing vizTrace2 envelopes (per-round topo refs, identity, containment, validation). |
| racket/prologos/tools/viz-capture-probe.rkt | Capture-scale probe tool for measuring trace/topology/round magnitudes. |
| racket/prologos/tests/test-viz-export.rkt | Golden/regression test for vizTrace2 envelope structure + soundness invariants. |
| racket/prologos/tests/test-trace-serialize.rkt | Update expected well-known cell counts in topology serialization tests. |
| racket/prologos/tests/test-propagator.rkt | Update well-known cell allocation expectations (next cell-id offsets). |
| racket/prologos/tests/test-propagator-bsp.rkt | Add Tier-1 observer coverage tests (fire-once flush now observable when armed). |
| racket/prologos/tests/test-preduce-ingest.rkt | Adapt tests to new routing model and dispatch-as-stratum handler. |
| racket/prologos/tests/test-preduce-egraph.rkt | Add parity harness for on-network WHNF drivers (loop + scheduler-driven). |
| racket/prologos/tests/test-observatory-01.rkt | Update expected auto-generated cell labels due to shifted well-known ids. |
| racket/prologos/rule-dispatch.rkt | Export/register process-dispatch-requests stratum handler. |
| racket/prologos/reduction.rkt | Add/route default WHNF through scheduler-driven on-network cascade; implement whnf-step1 + drivers and parallel demand. |
| racket/prologos/propagator.rkt | Add dispatch/reduce request well-known cells; make Tier-1 observable when observer is armed. |
| racket/prologos/driver.rkt | Remove PREDUCE_INGEST env toggle wiring (on-network routing is unconditional now). |
| racket/prologos/benchmarks/micro/bench-preduce-ingest.rkt | Remove ingestion overhead microbench (old gate-based comparison). |
| racket/prologos/eclass-graph.rkt | Add optional containment capture via current-eclass-containment-box. |
| racket/prologos/examples/2026-06-14-reduction-showcase.prologos | Example program showcasing on-network reduction + parallelism. |
| racket/prologos/examples/2026-06-14-parallel-reduction.prologos | Example emphasizing parallel operand reduction structure. |
| racket/prologos/examples/2026-06-14-onnetwork-reduction.prologos | Acceptance-style example exercising β/ι/δ/int-folds under on-network routing. |
| racket/prologos/examples/2026-06-14-fib-small.prologos | Small fib example for traceability under on-network reduction. |
| racket/prologos/examples/2026-06-14-fib-naive.prologos | Larger naive fib example for memoization/behavior exploration. |
| racket/prologos/examples/2026-06-12-ptf-track2-viz.prologos | Visualization acceptance corpus covering elaboration + relations + NAF. |
| docs/tracking/preduce-autonomy/HANDOFF.md | Update autonomy handoff status/context to reflect completed viz arc + pause. |
| docs/tracking/preduce-autonomy/dailies/2026-06-12_dailies.md | Add detailed daily log for the viz/export arc iterations. |
| docs/tracking/MASTER_ROADMAP.org | Add “Track V” entry for browser viz + trace export. |
| docs/tracking/DEFERRED.md | Record deferred viz riders (component diffs, deeper value detail). |
| docs/tracking/2026-06-14_PTF_VIZ_ROADMAP.md | Add roadmap describing progression toward “computation as propagation” viz goals. |
| docs/tracking/2026-06-14_PREDUCE_T8_PHASE4b5_NETWORK_DRIVEN_REDUCTION.md | Add Track 8 design/implementation notes for network-driven reduction phases. |
| docs/tracking/2026-06-14_PREDUCE_T8_PHASE4_RECURSION_ON_NETWORK.md | Add Track 8 notes for recursion recording + perf improvements. |
| docs/tracking/2026-06-14_PREDUCE_T8_PHASE2_DISPATCH_AS_PROPAGATOR.md | Add Track 8 notes for dispatch-as-stratum firing design/impl. |
| docs/tracking/2026-06-14_PREDUCE_DEFAULT_AND_ONNETWORK_COMPUTE_PLAN.md | Add plan document tracking phases toward default on-network reduction. |
| docs/tracking/2026-06-12_PTF_TRACK2_PIR.md | Add PIR documenting viz/export track outcomes and follow-ups. |
| docs/tracking/2026-06-12_PTF_TRACK2_BROWSER_VIZ_DESIGN.md | Add design doc for browser viz + trace export, including branch directive context. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+6
to
+11
| const fs = require('fs'), path = require('path'); | ||
| const html = fs.readFileSync(process.argv[2], 'utf8'); | ||
| const m = html.match(/\/\* @PURE-BEGIN \*\/([\s\S]*?)\/\* @PURE-END \*\//); | ||
| if (!m) { console.error('PURE block not found'); process.exit(1); } | ||
| eval(m[1]); | ||
|
|
Comment on lines
+3
to
+8
| const fs = require('fs'); | ||
| const html = fs.readFileSync(process.argv[2], 'utf8'); | ||
| const m = html.match(/\/\* @PURE-BEGIN \*\/([\s\S]*?)\/\* @PURE-END \*\//); | ||
| if (!m) { console.error('PURE block not found'); process.exit(1); } | ||
| eval(m[1]); | ||
|
|
…64 review) Addresses two Copilot review comments on PR #64: - viz-export.rkt: value-detail-for-network took the first N of (hash-keys v) without sorting; hash-keys order is unspecified, so the exported key list could vary run-to-run for the same program. Sort the stringified keys before truncation. Verified: two exporter runs are now byte-identical except for the observer's timestampMs/wallMs telemetry (which is intentionally wall-clock). - check-parens.sh: when the pinned binary is absent and racket is not on PATH, $RACKET went empty and each file produced a confusing exec error. Fail fast with an explicit message (exit 2). https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
Top-left source panel (toggle: 'code', default on) renders the program with lightweight Prologos syntax highlighting. Propagators carry their install-site srcloc line, so: - hovering a source line highlights its propagator nodes in the graph - hovering a node highlights its source line Lines with at least one propagator in the current round's topology are marked (green line number). Cells don't carry srclocs, so only propagator nodes map; the mapping is line-level (the exporter inherits the install-site srcloc, which is coarse — e.g., all fib-body reductions share the defn line). https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this branch does
This prototype branch builds a browser visualization of propagator-network execution for Prologos programs, and — so the visualization has something faithful to show — lifts term reduction onto the propagator network, so the viz renders genuine on-network reduction rather than a recording of an off-network evaluator.
Two intertwined arcs:
1. Browser visualization (
tools/viz/)tools/viz/index.html): live d3-force layout, layered mode, per-round playback (fired propagators highlight, written cells flash), e-class collapse, infrastructure filter, scheduler sparkline, and a node-label toggle.tools/viz-export.rkt) producing thevizTrace2envelope — per-command captures with per-round topology, cell diffs, and propagator-fire records.viz-export-expr-networktraces the reduction cascade for a single expression.2. On-network reduction (PReduce Track 8)
whnf-step1: a one-step classifier returningwhnf | native | (step C) | (demand …), covering β, projections, ι/natrec, J, boolrec, ann, vector head/tail, integer arithmetic, structural reduce, δ, and meta.Architectural limits (documented, not bugs)
whnf-implis a reimplementation rather than a mechanical migration — it is kept as the compute leaf.Verification
test-preduce-egraph): on-network whnf == native whnf.This is an experimental prototype branch; a performance regression from routing reduction through the network is expected and accepted at this stage.
https://claude.ai/code/session_016Pdxg4BhN6W9Cv6mXYDhUp