Name proof endpoints by value, and stop tabling reflexive proofs - #5
Closed
oflatt-claude wants to merge 336 commits into
Closed
Name proof endpoints by value, and stop tabling reflexive proofs#5oflatt-claude wants to merge 336 commits into
oflatt-claude wants to merge 336 commits into
Conversation
…ad-under-2x Reduce proof overhead with native tuple UFs and rule-name hoisting
…ark-coverage Expand benchmark coverage
Foundation for the relation-based term/proof encoding: - get-fresh! per-sort mint primitive (from the backend id counter) - term tables: constructor -> relation (function ... Unit :no-merge), created via (let fresh (get-fresh!)) (name children fresh) - extraction cost moved off term tables onto views via :internal-cost Terms functionally working (term-mode file tests pass modulo snapshots). Proof constructors still to convert. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- proof-datatype + AST + prooflist constructors -> relations (function ... Unit) - shared mint() helper (get-fresh! + set), returns the fresh var - term_proof_for_justification emits mint sequences, returns proof var Scattered nested proof-construction sites still to flatten. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every generated nested proof/AST/ProofList expression now builds bottom-up via mint() (get-fresh! + relation assert). format_prooflist emits mints. Term-mode parses/typechecks; proof-mode extraction still to update. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- extract_eq/prove_exists read the minted id from the last input column (is_relation_term / extraction_output_index) - prove-exists expects a function (constructors are lowered to term relations) - fix custom-function merge: bare-var merge result no longer wrapped into a call - container-rebuild proof primitive mints+asserts proof relation rows instead of constructor lookup-or-insert proof_mode_regression: 9/9 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduces per-view set-if-empty + view-proof primitives (proof_fresh.rs) and threads freshly-built constructor terms through set-if-empty so parents build with canonical children, keeping the FD views canonical (nothing for the encoded rebuild to re-key). Term-only mode: validated win on math-microbenchmark — @Rebuilding search+apply 2.67s -> 2.08s (-22%), total 4.45s -> 3.73s (-16%), tuples -10%. No new regressions (term suite 98/2, both pre-existing relation-encoding snapshot regens). Proof mode: canonicalization enabled but WIP. Deduped terms leave orphan term-relation rows (acceptable); the remaining work is bridging the canonical e-class back to a consumer's expected enode form via the view proof (Sym/Trans) in prove-exists / remove_globals. Proof suite currently 151/37. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the plan's natural-term + canonical-term + Congr + set-if-empty threading in add_term_and_view: builds the natural term (children at as-built ids) and canonical term (children at view-deduped ids), connects them with a Congr chain over changed children, dedups via set-if-empty, and records (natural, connector: natural = deduped) in nat_conn for the parent's Congr. Returns the deduped e-class so parents/views stay canonical. Fixes some proof tests (e.g. eqsat_basic). Remaining: the root union still builds its Rule-proof AST from the deduped e-class (whose AST floats to a unioned form), so it must consume the connector (Trans of rule[L=nat] with connector[nat=dedup]); the UF-edge orientation makes that intricate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The root union now routes its proof through the operands' natural forms (whose ASTs are pinned to the enode the rule built) instead of the deduped e-classes (whose ASTs float once unioned), then orients to the larger=smaller UF edge with proof-of-max/min over a shared natural form. Fixes proof checking for the term-construction plan (the (Neg (Add a (Add b c))) example proves cleanly) and simple rewrites. Remaining: complex/saturating rewrites where the *matched* LHS (a body var, no natural form recorded) has itself been unioned and its AST floats. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…en term The canonical-children term (fv_can) was given a fresh @rule term proof, which the checker validates against the rule head via check_rule_produces_equality. But fv_can is built with *deduped* children, which can extract to a canonicalized shape (a subterm rewritten elsewhere, e.g. matrix's nrows(NamedMat B) -> NamedDim m) that no longer matches the syntactic rule head -> 'rule head doesn't produce claimed equality'. Per the plan, fv_can's proof should be the reflexive Trans(Sym(e-to-e'), e-to-e') derived from the Congr chain, not a @rule -- so it is exempt from the rule-head check and serves as the view's 'eclass = f(children)' proof. The natural term keeps its @rule (its children are the as-matched forms, which do match the head). Fixes matrix/naturals/fibonacci proof-testing; 'rule head' failures 14 -> 2 (remaining: luminal complex rule + pre-existing @p1 custom-merge). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…erms Root cause of the proof-mode canonicalization failures on deeply-nested rules (e.g. matrix, and the constructor parts of luminal/hardboiled_conv1d): instrument_action's Let case bound the let variable to the built term's deduped e-class var but did NOT carry the term's nat_conn entry onto the let name. So a rule like `(let new-e (Bop ...)) (union e new-e)` looked up nat_conn["new-e"], missed, and `union` fell into the no-connector branch: a bare @rule over ordering-max/min(e, new-e) whose endpoint extracts the *deduped* (canonicalized) shape, while process_actions builds the syntactic shape -> "rule head doesn't produce claimed equality". Fix: propagate nat_conn from the result var to the let-bound name. Also keep the natural node *unseeded*: always mint a separate canonical-children node to seed the view (never collapse to seeding fv_nat), so the natural is never pulled into the view's congruence :merge (native UnionId) and stays as-built. Otherwise a birewrite re-keying the view to a differently-shaped partner (IntImm32 x <-> IntImm 32 x) natively merges the natural into the partner and its @rule endpoint extracts the wrong shape. Result: matrix, naturals, fib and all other constructor-based proofs prove cleanly. Remaining: luminal still fails on its `vec-of` container (containers are built with deduped elements and their @rule extracts the canonical shape rather than threading the natural like constructors -- a separate follow-up), and the pre-existing @p1 custom-merge bug is unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The connector-threading fix changed the instrumented proof structure, so all proof-testing snapshots for passing constructor-based proofs are regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nat_conn maps an instrumentation unit's freshly-minted vars (and let-bound
names) to their natural id + connector; it is local to a single generated
program. The connector-threading fix keys it by the user's let-variable names,
which repeat across rules, so without resetting it, stale entries leak between
rules/merges and can reference out-of-scope vars ("Unbound symbol @vn"). Clear
nat_conn at the start of each rule, merge function, and global action.
No current test changes result; this prevents cross-unit leaks (which a later
container change would otherwise surface).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes luminal's vec-of container proof failure with the container rebuild itself, rather than an action-side connector. A container built over deduped element ids has a term-proof whose endpoint extracts the canonicalized (birewrite-partner) shape instead of the syntactic one -> "rule head doesn't produce claimed equality" for a parent term. Building over the *natural* (as-built) element ids fixes the proof, but naturals have no union-find entry, so the container rebuild can't canonicalize them and the computation breaks. Fix: add a per-eq-sort auxiliary union-find `@UF-Aux-<Sort>` mapping a natural id to its canonical dedup id plus the connector proof `natural = canonical`. The container is built over natural element ids (so the term-proof extracts the syntactic shape); at build time each element's `natural -> (canonical, connector)` edge is written to `UF-Aux`. The Rust container rebuild (`rebuild_container_value_rec` / `rebuild_container_proof_rec`) now consults `UF-Aux` in addition to the main `UF`, chaining `natural ->(UF-Aux)-> canonical ->(UF)-> leader` and composing the connector with the UF proof via `Trans`. So the rebuild canonicalizes the elements and proves it with the existing machinery. `UF-Aux` is written only for container elements. Deterministic table name (`@UF-Aux-<Sort>`) so the rebuild needs no extra plumbing. Natural-element building is applied to `vec-of` only: it makes elements change during rebuild, which triggers the rebuild's per-element `@Congr` fold, and that fold is positional (sound only for ordered containers). Sets/maps keep the deduped path; an order-independent rebuild proof for a birewrite-canonicalized unordered-container element is a follow-up (no current test needs it). Result: luminal (both variants) and all container tests prove cleanly, with no proof-snapshot changes; only the pre-existing @p1 custom-merge failures remain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`reflexivize_premise` built `Trans(Sym(p), p)` with bare `id_to_proof` pushes while `proof_head` builds the same thing through `push_shared_proof`, so the two rows of one firing could reflexivize a premise to different ids — and the premise vector is both the head-walk memo's key and a rule proof's sharing key. `ProofAlgebra::reflexive` is that composition, over the shared builders. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`HeadProof::GlobalValue` was the one proof a head's walk filled with `None` and the one no encoder site named, so every head containing a `set` carried a padding column. A `set` claims its row and nothing else. `HeadPosition::Set` stays distinct from `Call`: the layout assertion is what catches a `set` lowered where the walk expects a call. Encoding change: a numbered column after a `set` shifts down by one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rebuild skeleton was written twice — once where the encoder builds the row, once in the test that unpacks it — so the test round-tripped its own copy of the shape rather than the one the encoder writes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Skeleton` and `Composition` were variant-for-variant identical; the only difference was what a leaf holds, and `Composition::pack` is the map between them. One `ProofTree<L>` says that, and gives `Composition` the `sym`/`trans`/`congr` builders it lacked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every constructor's arity and child positions were written twice, in `parse_proof_inner` and again in `nested_proofs` with a different scrutinee. A constructor added to one and not the other left only a `debug_assert`: in release, a deep chain through it overflows the stack instead of reporting anything. One table says how a constructor is read, and both derive from it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- `RawProofStore::proof_to_term` was written on every parse and read nowhere; its doc called the pair a bidirectional map, which it is not. - `compute_rule_substitution`'s length check is unreachable: its one caller asserts the premises cover the body and then zips, which truncates to exactly that length. - `unify_fact` has one caller, in the same `impl`. - `is_custom_func_fact`'s reversed `Eq(Var, Call)` arm cannot match: proof normal form always writes the call on the left. - `stage_batch`'s row count is always positive — `run_instrs` returns on an empty mask and neither `Insert` nor `Remove` clears it — so the count, the return, and the branch on it do nothing. - `source_at` and `row_sources` re-matched `ValueSource` and `QueryEntry`, which `mask.rs` already does; both now live there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The changelog entry described the encoding's internals -- which premises ride in which row, what a rebuild packs. None of that is visible to someone using egglog: the proofs are identical. What changed for them is that proof mode is faster and smaller. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`site` and `head_chain` lived on `ProofInstrumentor` and were scoped by assignment: set on entering a head, cleared on leaving it, saved and restored around a `change` argument. A missed reset leaked one rule's numbering into the next. They are now one `Head`, threaded through the action instrumentation, so the borrow is the scope. Every block that lowers actions makes its own — a rule head, a top-level action, a merge body, an `extract` expression — and the two that used to run on whatever the last head left behind now say so. The chain loses its `Option`: there is no longer anywhere without a head to hold it. The emission state stays on the instrumentor. `reflexive`, `deferred` and `sealed` are filled by `instrument_facts` and read by the head that follows, so they outlive a head; all four, `packed_decls` included, are reached through `mint`, which every declaration and rebuild rule also calls. Threading the head pushed four functions past the argument limit, so each gives one up: a `union` claims its own columns, a composed union edge orders its own endpoints, a term looks up its own e-class sort, and a construct-into guest is bound by its caller like any other `let`. The encoding is byte-identical over all 109 encodable tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seventeen functions took some of `stmts`, `head` and `justification`, and fourteen took all three. Those three are one thing — where a row goes, which head it belongs to, and why it holds — so they are now one `Emit`, and the functions that lower an action take that instead. Nothing in the file passes more than five arguments now; six functions used to pass seven. Two of `Emit`'s methods say what the extra arguments used to. `justified_by` gives the same place a different reason: a numbered column of the same rule, or a proof composed on the spot, so a composed union edge says "fiat" once at the top instead of at each of its four calls. `composing` runs a nested position the head concludes nothing about. `scope` stays out of it. It is the binding environment an expression reads, not a place to write; only three of these functions take one, and `instrument_action` needs it `&mut` where the others do not. Two of the functions were long not for want of a struct but because they had a second way out: `add_term_and_view` and `instrument_action` returned a `Vec<String>` that every caller immediately extended onto the buffer it had already handed them. They write to the one buffer now. Threading the head cost four arguments last commit, and all four still stand without the pressure. Three positions are claimed by the function that knows which position it is — a `union`, a construct-into guest, a call — and only `add_term_and_view`, shared by four of them, is told; a composed union edge orders endpoints it already holds both of; `term_sort` names the lookup three constructor builders were spelling out, and now a fourth; and a guest binds its `let` in the caller, like the ordinary `Let` arm beside it. The encoding is byte-identical over all 109 encodable tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An index is probed rather than scanned, so the value it is looked up by has to be bound elsewhere in the query. The check treated every other index atom as binding nothing, which also rejects the case where one index's *row columns* bind the value a second index is probed by -- a query core-relations' own occurrence-var check accepts. Only the leading argument, the value an atom is itself probed by, is excluded now; two index atoms naming each other there would each look bound with neither reachable. Tested both ways: restoring the old exclusion fails the accepting case and leaves the rejecting one passing. Reported by Copilot on saulshanabrook#39. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An index atom is the only body atom the DD backend did not count as one, so a rule whose body is nothing else was partitioned as atom-less: fired once from an empty environment, its head reaching for variables the join never bound. `plan_join` lowers that body fine, so the rule failed with "unbound term in rule head" rather than producing its match. A declared index registers a function type but no backend table, so a command that looks one up by name found nothing where it expected a table: `(input <index> …)` unwrapped a `None`, and an index over another index reached the missing entry when a rule first queried it. Both now report the index rather than panicking. Also: bound an occurrence index's columns by the function's own schema rather than the backing table's, which includes the timestamp and subsume columns; validate an index atom's terms as the other body atoms are; walk only the set bits of an occurrence column mask; and say in the backend SPI that an index atom carries the index relation's unit output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two statements per node, and two term nodes per `t = t`, were most of what the encoder emitted. The top-level block for `(Add (Num 1) (Num 2))` was 44 statements for three constructor applications; it is now 25. A term/proof/AST node is written by `(let v (mint-<Relation>! args…))`, one call that mints the id and asserts the row, in place of a `get-fresh!` binding and a `set`. Every relation whose last input is a minted id — the term relations, the `@Ast<Sort>` wrappers, every proof-node relation — gets one, registered where its declaration is typechecked, beside the view's `set-if-empty`, so the desugared program still re-parses. A per-relation family rather than one generic op: the relation is then in the signature, so the call typechecks and resolves like any other primitive, with no table name to decode at run time. Backends service it through `Backend::register_mint_row`. A merge body that mints must still declare the write, or the strata that order `merge_all` can put the minting relation in the same wave as the merging one and leave it out of the read-only view. The bridge records the table each row-inserting external function writes and `MergeFn::fill_deps` declares it, which is the dependency the `set` used to carry. A reflexive `Fiat` now names one `@Ast` node on both sides instead of minting two over the same value. Nothing downstream changes: the two sides were already extracted to the same hash-consed term, so `RawProof::Fiat`, proof conversion and the checker all see what they saw before. Encoding every `egglog/tests` program in proof mode: 199,408 statements to 153,100 (-23%), the same 109 files encoding and no file growing. Rows per firing are unchanged (2 for a flat rewrite, 6 nested, 1 per view rebuild, 1 per merge collision); `(Add (Num 1) (Num 2))` keeps its 4 proof rows and goes from 6 `@Ast` rows to 3. The one snapshot that moves is `doc_example_add_function1`, which dumps the encoded program; its diff is two fused mints and nothing else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The proof system had no way to get from a proof about `f(a, b, c)` to a proof about `a`, so the only way to say a term had been built was to look its reflexive proof up in `@<Sort>Proof`. Add the projection rule the mechanized system has: from `t = f(..., ci, ...)` and the index of `ci`, conclude `ci = ci`. It is stated at the equality level rather than as a separate "is built" judgement, because in egglog `t = t` already is that judgement -- reflexivity is not assumed, so a proof of it is exactly a witness that `t` was built. That collapses the mechanized system's projection and reflexivity rules into one. The encoder declares `@Proj` but writes none: every reflexive anchor it needs it already holds, and deriving one instead costs a minted row where the table costs a lookup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`@<Sort>Proof` held a `t = t` proof for every term and every container — 11,972 writes over the corpus, 15% of every statement the encoder emits — so that anywhere a reflexive anchor was wanted it could be looked up. Every one of those places already has a row proof in scope whose right-hand side mentions the term, so the anchor is a projection out of it: `Proj` at a child position, `Trans`/`Sym` for a constructor view's e-class, and a new raw-only `ProjAll` — `CongrAll`'s projection counterpart, desugared to `Proj` during conversion — for a container element, whose position in the term form is only known once the term is in hand. Which row anchors a rule body's variable is only known once the whole body is walked, so the body collects what each view atom offers and binds the anchors at the end; a value a body primitive reads out of a container has no anchor at all and becomes a side condition the checker re-evaluates, alongside the container-producing primitives that already were. Conversion now binds a side condition's output by the same evaluation, so a rule head may use it. With nothing left reading a term's own conclusion, a rule head writes no row for it either. The corpus goes from 77,101 statements to 58,493; a flat rewrite writes 1 proof row where it wrote 2, the nested head in proof_encoding.md 3 where it wrote 6, and a view rebuild one packed row plus one anchor per canonicalized column where it wrote one packed row. `web-demo/math`, `herbie` and `typeinfer` run in 0.90x, 0.89x and 0.92x their wall time. The proofs that move are anchors changing provenance, never a proof changing shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`uf_canon_proof` answers a rebuild's "did this column move?" with the column's `@UF` row proof, or, on the miss that says it did not move, with the column's reflexive anchor. The caller minted that anchor eagerly and passed it in, so every firing wrote one anchor row per canonicalized column and threw away the ones whose column had moved — four rows per firing for a binary constructor where one row is the point. The primitive now takes what the projection needs, the row proof and the column's position, and mints `Proj(row_proof, position)` itself on the miss. Backends service the composed op through the new `Backend::register_view_column_read_or_mint`, so it stays a view-column read the backend answers against its own storage rather than a registry primitive the Differential Dataflow backend cannot run. A constructor view's e-class is the row proof's left-hand side rather than one of its children, so it takes the projection position -1, which parsing reads back as the reflexivization `Trans(p, Sym(p))` the rebuild used to spell into a `@Packed_1` row. Same proof, same node, written one call later. The container rebuild keeps its eager anchor: it composes from that anchor on every call, so there is no miss to defer to. A firing of a binary constructor's rebuild goes from 4 proof rows to 1 plus one anchor per column that stayed put, and the corpus from 58,493 statements to 56,263. `web-demo/math`, `herbie` and `typeinfer` run in 0.97x, 0.99x and 0.95x their wall time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`(= (vec-get ys 0) (vec-get ys 1))` is settled by the primitives alone, but the side-condition gate demanded a literal variable on one side, so the fact took the premise path and asked for a reflexive anchor on each element. An anchor is a projection out of a row proof the body reads, so a container the body *built* — `(= ys (vec-of ...))` — anchors nothing, and the request panicked. Its variable-shaped sibling `(= e (vec-get ys 0))` was already a side condition and already worked. Admit primitive result against primitive result too. The fact routes to `check_side_condition`, whose both-sides-determined branch re-evaluates each side and compares — which is what verified the fact before as well, since a premise composed from two reflexive anchors reduces to a reflexive proof and `check_fact_matches_proposition` then requires both sides to agree with it. The encoder emits the fact as-is with the `Eval` marker and requests no anchor. A built term against an element read, `(= (Add a b) (vec-get v 1))`, stays out: it asserts an e-graph equality re-evaluation cannot settle, and keeps its real premise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tayed put" This reverts commit d3fb7e1. Deferring an anchor to the `@UF_<Sort>` miss cut the corpus by 3.8%, but moved neither wall time (0.96-0.99x, within noise) nor peak RSS (49/49, 65/61, 36/36 MiB on web-demo/math, herbie, typeinfer). It cost a new backend SPI method threaded through egglog-bridge and the DD backend's action and merge paths, and a sentinel projection position standing for a reflexivization. A smaller emitted program is not on its own worth that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the checker Anchoring a term by projecting it out of a row proof left two body shapes with no row to project from, and the gate the checker uses to skip a premise was widened twice to cover them: an eq-sort value read out of a container the query itself built, and one a primitive produced without being handed a container at all. The checker is the trusted base, so it should not move to accommodate the encoder. Restore it to the projection rule and nothing else, and reject those two shapes up front instead. A container's reflexive proof cannot be derived either: `@Congr`/`@CongrAll` rewrite a child of a term some proof already mentions, so neither can introduce the container's term in the first place, and `@Fiat` is reserved for literals and value-constructor terms. There is no introduction rule for a term the database never held, which is the point -- a value the query computed is one proofs have nothing to say about. Two new `ProofEncodingUnsupportedReason`s say so, alongside the query-built container an action may not carry. `container-proofs.egg`'s one affected rule moves to `container-query-built-read.egg`, which is expected-unsupported, so the rest of the file keeps encoding under proofs. A body's equalities now join the variables they relate into one anchor class, so `(= e (vec-get v 0)) (= f (vec-get v 1)) (= e f)` over a matched container is anchored through the reads rather than asking after `f` alone. Those premises go back to a real `@ProjAll` off the container's row where the widened gate had left them an `@Eval` marker. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two rejections added with the projected anchors fired on the shape of a body fact: any eq-sort value read out of a container the query built, and any eq-sort value a primitive produced with no container to read it out of. Both are broader than the encoder is. A fact's premise composes `Sym(left)` with `right` and drops whichever side proves `t = t`, so an equality reads its right-hand anchor exactly when its left-hand proof is itself reflexive. A view atom's argument, the right-hand side of a term the query builds, and a value another atom anchors after all all encode fine today, and were being turned away. Scan the body for what the encoder's `BodyAnchors` would collect -- which values a view row reaches, which equalities alias them, which reads came out of which container -- and reject only a premise that reads an anchor no view row supplies. Which value that is decides the reason, so both keep their meaning: a container read names the container, and one with no container names the primitive. Note that it is the premise, not the actions, that reads such an anchor: the premise of the fact binding the value *is* its reflexive proof, so removing the value's use from the head changes nothing. The rejection tests say so now, and `container-query-built-read.egg` stays unsupported for that reason. A head that mints no row reads no premise at all, so those rules would encode too. Proof support does not model which actions read the rule proof and rejects them, which is the safe side of the invariant the encoder asserts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every encoded constructor carried a deferred-delete and a deferred-subsume relation, a rule for each, and one marker-rebuild rule per eq-sort column, whether or not the program ever deleted or subsumed. Only 46 of 696 corpus constructors ever do. The delete indirection bought nothing. The backend already stages a removal to the end of the batch, and a table's commit applies removals ahead of insertions, so lowering `delete` straight to the view row reproduces the uninstrumented meaning. The marker instead delayed the removal to the next maintenance pass, which deleted a row a rule inserted in the same batch, missed a row already subsumed, and missed a row the intervening rebuild had re-keyed. Subsumption still needs its marker, because rebuilding re-keys a view row by inserting it afresh and the new row carries no subsumed bit. Its scaffolding is now emitted ahead of the first command that subsumes the constructor. Commands arrive one at a time, so a pass over the program cannot find them; the memo lives on the e-graph, which `push` clones and `pop` restores along with the declarations it tracks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The container rebuild minted an element's `@Ast` node and its `@ProjAll` anchor before knowing whether the element changed, then discarded both — along with the whole recursive proof standing on them — whenever it had not. Ask the value-level rebuild first, as the eq-sort branch beside it already does, and mint nothing for an element that stays put. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This reverts commit cc807d5. Deferring a nested container element's anchor until the element is known to have moved is correct, and on a purpose-built (Vec (Vec E)) probe it fires. On the corpus it cannot: @ProjAll holds zero rows on both eggcc-2mm and hardboiled_conv1d_32, so the nested-container branch is never entered. Wall time and peak RSS are unchanged within noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A `@Fiat` named its two endpoints by wrapping each in an `@Ast<Sort>` row, whose only job was to erase the endpoint's sort so one relation could take both. Specializing the relation on the sort does the same erasure at the declaration instead: `@Fiat_<Sort> (S S @proof)` names the values directly, so a reflexive fiat costs one row rather than two. The declaration goes with the first command needing it, like `@Packed_<k>` and `@to_subsume_<Ctor>`, and the name is derived from one prefix so a re-parsed program recovers it from the `@Proof` sort's `:internal-proof-names` without having encoded anything. eggcc-2mm: 2.081M -> 2.005M rows (-3.6%), 17 `@Fiat_<Sort>` declarations against 120 `@Ast<Sort>` tables of which those same 17 ever held a row; forms 2722 -> 2738, RSS 397 -> 388 MiB, wall unchanged. hardboiled_conv1d_32: 345.8k -> 328.4k rows (-5.0%), forms 1035 -> 1049. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`@ProjAll` named the projected child through an `@Ast<Sort>` row, the last reader of the wrapper. Specializing it the same way the fiat justification now is — `@ProjAll_<Sort> (@proof S @proof)` — names the child directly, and the `@Ast` sort, its per-sort wrappers, their mint primitives, the `:internal-ast-func` annotation and `unwrap_ast` all go with it. A container sort's projection is declared with the sort rather than on first use: the rebuild primitive mints one for a nested container without going through any statement the encoder wrote. Proof mode is now read off the `@Proof` sort being declared, which is what `:internal-ast-func` was standing in for at `register_uf_canon`. eggcc-2mm: 2722 -> 2617 top-level forms across the two commits (-3.9%), of which 121 go here (120 wrappers plus the sort). hardboiled_conv1d_32: 1035 -> 1003. Rows and wall unchanged here — the wrapper's rows went with the fiat change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A body fact over base values — a guard like `(> n 0)`, or the `(= len n)` a custom function's output leaves behind — has a reflexive fiat for its premise, whose term is a function of the fact and the bindings the earlier facts made. Conversion can re-evaluate it, so the encoding writes neither the fiat row nor the premise column that would name it, and the firing's `@Rule_<k>` is one column narrower. One gate over the body decides which facts these are, shared by the encoder and proof conversion so the two cannot drift, and taken in body order because a fact is only recomputable once one of its sides reads nothing but already-bound variables. Conversion therefore walks the body once, interleaving premise conversion with the substitution it feeds, instead of converting every premise up front. The checker is untouched: it already pins these propositions, since it re-evaluates each fact against the full substitution to match it. eggcc-2mm: 2.004M -> 1.930M rows (-3.7%), 75.1k `@Fiat_<Sort>` rows down to 567. hardboiled_conv1d_32: 329k -> 308k (-6.4%), 18.7k down to 980. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sweep the crate for prose describing machinery this branch deleted: the per-sort reflexive proof tables, the AST wrapper sorts, and the deferred deletion markers. Fold the dangling doc block the removed `compute_rule_substitution` left above `rule_premises` into that function's own contract, and trim the doc comments this branch added down to what a caller needs. Clear `unanchored` with the rest of the held-back state, so a body whose anchor goes unread does not keep the deferred-lookup fast path off for every later rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner
Author
|
Superseded by saulshanabrook#48, opened against upstream. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Name proof endpoints by value, and stop tabling reflexive proofs
The proof encoding used to reach a term's reflexive proof (
t = t) by looking it up. Every sort got a@<Sort>Proof : (S) -> @Prooftable, every constructor wrote one row into it per node built, and every sort also got an@Ast<Sort>wrapper so a@Fiatcould name endpoints of any sort through a single erased@Astsort. Both are now gone.A reflexive anchor is instead projected out of a row proof already in scope. A view row's proof states an equality whose right-hand side is the row's term, so a child of that term is
@Proj(row, i), the row's e-class is the row proof reflexivized, and a container element — whose position in the term form is only known once the term is in hand — is@ProjAll_<Sort>(row, element).@Projis the branch's one new checker rule;@ProjAll_<Sort>is raw-only and conversion desugars it to the@Projat the position it finds, exactly as it already did for@CongrAll. With endpoints named by value,@Fiat_<Sort>and@ProjAll_<Sort>are specialized per sort and the@Astsort and its per-sort wrappers disappear.Three further reductions in what a firing writes:
mint-<Relation>!primitive, so(let v (get-fresh! "Math")) (set (Add a b v) ())becomes(let v (mint-Add! a b)). Registered through the backend SPI (Backend::register_mint_row), with the DD backend servicing it against its mirror.deletelowers directly. The@to_delete_<F>marker table and its maintenance rule are gone; the action deletes the view row. The backend already stages removals and applies them ahead of the same batch's insertions, which is the uninstrumented meaning ofdelete, so lowering it directly preserves it — the marker did not, and diverged from nativeegglogin 4 of 6 adversarial cases (deleting rows a same-batch insert should have kept; failing to delete rows that were subsumed or that rebuild had re-keyed). Subsumption stays deferred (its marker has to survive re-keying), but its scaffolding is now declared on firstsubsumerather than with every constructor.(> n 0), or the(= len n)a custom function's output leaves behind — is re-evaluated by proof conversion from the bindings earlier facts made. Encoder and conversion share one gate (recomputable_premises) so they cannot drift.The proof checker is unchanged from the base of this branch apart from the
Projrule:git diff <base> HEAD -- proof_checker.rsis 54 insertions, 0 deletions.Measured effect
Root
./bench.py,mainbackend, 12 rounds, default workload set. An A/A control (same source, two builds) put the noise floor at ±1% per file on min-of-12 wall and ±0.1% on peak RSS; deltas below that are not reported as signal. Wall deltas are min-based, reproduced across two independent passes.Proof mode — suite wall 0.806x (−19.4%), peak RSS −25.9%:
Most of the win is program construction, charged outside rulesets: −1.71 s of eggcc-2mm's −2.0 s, −2.53 s of luminal-llama's −2.56 s. That is the deleted tables and the declare-on-first-use scaffolding shrinking the generated program.
math-microbenchmarkis a genuine regression, reproduced in both passes and outside the noise floor. Phase attribution: Apply +615 ms, Merge −391 ms — the branch moves work out of merge into apply on this workload for ~150 ms net. Its RSS still improves 19%.Native backend — neutral. Suite +0.2%/+0.3% against an A/A control of −0.2%; every per-file delta inside the floor, peak RSS identical.
Known limitation
Programs that were accepted under proofs are now rejected. A value the query computed has no row to project an anchor out of, and there is no congruence route to introduce its term. This only bites where a premise actually reads that anchor — a fact's premise composes
Sym(left)withrightand drops whichever side is reflexive, so the right-hand anchor is read exactly when the left-hand proof is itself reflexive. Two newProofEncodingUnsupportedReasons cover it:ContainerCreatedInQueryProvedAbout(an eq-sort element read out of a container the query built) andEqSortPrimitiveResultWithoutContainer(an eq-sort value a primitive produced with no container to have read it out of). One test file moved to the unsupported list (container-query-built-read.egg); base-sorted reads (vec-lengthand friends) over query-built containers are unaffected. A head that mints no row reads no premise either, but proof support does not model that and rejects those rules too.Notes for review
CHANGELOG.mdentry, deliberately — this is a follow-up arc and the entry belongs to the release that ships it.BodyAnchorScanin the support check,BodyAnchorsin the encoder). They reason about the same alias classes over different value representations and can drift; only anassert!catches it. Worth a follow-up.🤖 Generated with Claude Code