Skip to content

Name proof endpoints by value, and stop tabling reflexive proofs - #5

Closed
oflatt-claude wants to merge 336 commits into
mainfrom
reduce-program-blowup
Closed

Name proof endpoints by value, and stop tabling reflexive proofs#5
oflatt-claude wants to merge 336 commits into
mainfrom
reduce-program-blowup

Conversation

@oflatt-claude

Copy link
Copy Markdown
Owner

Name proof endpoints by value, and stop tabling reflexive proofs

The proof encoding used to reach a term's reflexive proof (t = t) by looking it up. Every sort got a @<Sort>Proof : (S) -> @Proof table, every constructor wrote one row into it per node built, and every sort also got an @Ast<Sort> wrapper so a @Fiat could name endpoints of any sort through a single erased @Ast sort. Both are now gone.

A reflexive anchor is instead projected out of a row proof already in scope. A view row's proof states an equality whose right-hand side is the row's term, so a child of that term is @Proj(row, i), the row's e-class is the row proof reflexivized, and a container element — whose position in the term form is only known once the term is in hand — is @ProjAll_<Sort>(row, element). @Proj is the branch's one new checker rule; @ProjAll_<Sort> is raw-only and conversion desugars it to the @Proj at the position it finds, exactly as it already did for @CongrAll. With endpoints named by value, @Fiat_<Sort> and @ProjAll_<Sort> are specialized per sort and the @Ast sort and its per-sort wrappers disappear.

Three further reductions in what a firing writes:

  • One statement per node. Every relation whose last input is a minted id gets a mint-<Relation>! primitive, so (let v (get-fresh! "Math")) (set (Add a b v) ()) becomes (let v (mint-Add! a b)). Registered through the backend SPI (Backend::register_mint_row), with the DD backend servicing it against its mirror.
  • delete lowers directly. The @to_delete_<F> marker table and its maintenance rule are gone; the action deletes the view row. The backend already stages removals and applies them ahead of the same batch's insertions, which is the uninstrumented meaning of delete, so lowering it directly preserves it — the marker did not, and diverged from native egglog in 4 of 6 adversarial cases (deleting rows a same-batch insert should have kept; failing to delete rows that were subsumed or that rebuild had re-keyed). Subsumption stays deferred (its marker has to survive re-keying), but its scaffolding is now declared on first subsume rather than with every constructor.
  • A base-value premise is recomputed, not recorded. A fact whose premise is a reflexive fiat over a base value — a guard like (> n 0), or the (= len n) a custom function's output leaves behind — is re-evaluated by proof conversion from the bindings earlier facts made. Encoder and conversion share one gate (recomputable_premises) so they cannot drift.

The proof checker is unchanged from the base of this branch apart from the Proj rule: git diff <base> HEAD -- proof_checker.rs is 54 insertions, 0 deletions.

Measured effect

Root ./bench.py, main backend, 12 rounds, default workload set. An A/A control (same source, two builds) put the noise floor at ±1% per file on min-of-12 wall and ±0.1% on peak RSS; deltas below that are not reported as signal. Wall deltas are min-based, reproduced across two independent passes.

Proof mode — suite wall 0.806x (−19.4%), peak RSS −25.9%:

benchmark wall Δ peak RSS
luminal-llama −31.1% 641 → 384 MiB (−40.1%)
hardboiled_conv1d_32 −21.6% 135 → 103 MiB (−23.7%)
eggcc-2mm-pass1 −20.0% 527 → 348 MiB (−33.9%)
pointer-analysis-small −19.7% at RSS floor
herbie −19.0% at RSS floor
math-microbenchmark +3.8% 1108 → 899 MiB (−19.0%)
suite −19.4% 2618 → 1940 MiB

Most of the win is program construction, charged outside rulesets: −1.71 s of eggcc-2mm's −2.0 s, −2.53 s of luminal-llama's −2.56 s. That is the deleted tables and the declare-on-first-use scaffolding shrinking the generated program.

math-microbenchmark is a genuine regression, reproduced in both passes and outside the noise floor. Phase attribution: Apply +615 ms, Merge −391 ms — the branch moves work out of merge into apply on this workload for ~150 ms net. Its RSS still improves 19%.

Native backend — neutral. Suite +0.2%/+0.3% against an A/A control of −0.2%; every per-file delta inside the floor, peak RSS identical.

Known limitation

Programs that were accepted under proofs are now rejected. A value the query computed has no row to project an anchor out of, and there is no congruence route to introduce its term. This only bites where a premise actually reads that anchor — a fact's premise composes Sym(left) with right and drops whichever side is reflexive, so the right-hand anchor is read exactly when the left-hand proof is itself reflexive. Two new ProofEncodingUnsupportedReasons cover it: ContainerCreatedInQueryProvedAbout (an eq-sort element read out of a container the query built) and EqSortPrimitiveResultWithoutContainer (an eq-sort value a primitive produced with no container to have read it out of). One test file moved to the unsupported list (container-query-built-read.egg); base-sorted reads (vec-length and friends) over query-built containers are unaffected. A head that mints no row reads no premise either, but proof support does not model that and rejects those rules too.

Notes for review

  • No CHANGELOG.md entry, deliberately — this is a follow-up arc and the entry belongs to the release that ships it.
  • Two independent implementations of the anchor gate remain (BodyAnchorScan in the support check, BodyAnchors in the encoder). They reason about the same alias classes over different value representations and can drift; only an assert! catches it. Worth a follow-up.

🤖 Generated with Claude Code

saulshanabrook and others added 30 commits July 14, 2026 14:25
…ad-under-2x

Reduce proof overhead with native tuple UFs and rule-name hoisting
Foundation for the relation-based term/proof encoding:
- get-fresh! per-sort mint primitive (from the backend id counter)
- term tables: constructor -> relation (function ... Unit :no-merge),
  created via (let fresh (get-fresh!)) (name children fresh)
- extraction cost moved off term tables onto views via :internal-cost

Terms functionally working (term-mode file tests pass modulo snapshots).
Proof constructors still to convert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- proof-datatype + AST + prooflist constructors -> relations (function ... Unit)
- shared mint() helper (get-fresh! + set), returns the fresh var
- term_proof_for_justification emits mint sequences, returns proof var
Scattered nested proof-construction sites still to flatten.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every generated nested proof/AST/ProofList expression now builds bottom-up
via mint() (get-fresh! + relation assert). format_prooflist emits mints.
Term-mode parses/typechecks; proof-mode extraction still to update.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- extract_eq/prove_exists read the minted id from the last input column
  (is_relation_term / extraction_output_index)
- prove-exists expects a function (constructors are lowered to term relations)
- fix custom-function merge: bare-var merge result no longer wrapped into a call
- container-rebuild proof primitive mints+asserts proof relation rows instead
  of constructor lookup-or-insert
proof_mode_regression: 9/9 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduces per-view set-if-empty + view-proof primitives (proof_fresh.rs) and
threads freshly-built constructor terms through set-if-empty so parents build
with canonical children, keeping the FD views canonical (nothing for the encoded
rebuild to re-key).

Term-only mode: validated win on math-microbenchmark — @Rebuilding search+apply
2.67s -> 2.08s (-22%), total 4.45s -> 3.73s (-16%), tuples -10%. No new
regressions (term suite 98/2, both pre-existing relation-encoding snapshot regens).

Proof mode: canonicalization enabled but WIP. Deduped terms leave orphan
term-relation rows (acceptable); the remaining work is bridging the canonical
e-class back to a consumer's expected enode form via the view proof (Sym/Trans)
in prove-exists / remove_globals. Proof suite currently 151/37.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the plan's natural-term + canonical-term + Congr + set-if-empty
threading in add_term_and_view: builds the natural term (children at as-built
ids) and canonical term (children at view-deduped ids), connects them with a
Congr chain over changed children, dedups via set-if-empty, and records
(natural, connector: natural = deduped) in nat_conn for the parent's Congr.
Returns the deduped e-class so parents/views stay canonical.

Fixes some proof tests (e.g. eqsat_basic). Remaining: the root union still
builds its Rule-proof AST from the deduped e-class (whose AST floats to a
unioned form), so it must consume the connector (Trans of rule[L=nat] with
connector[nat=dedup]); the UF-edge orientation makes that intricate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The root union now routes its proof through the operands' natural forms (whose
ASTs are pinned to the enode the rule built) instead of the deduped e-classes
(whose ASTs float once unioned), then orients to the larger=smaller UF edge with
proof-of-max/min over a shared natural form. Fixes proof checking for the
term-construction plan (the (Neg (Add a (Add b c))) example proves cleanly) and
simple rewrites. Remaining: complex/saturating rewrites where the *matched* LHS
(a body var, no natural form recorded) has itself been unioned and its AST
floats.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…en term

The canonical-children term (fv_can) was given a fresh @rule term proof, which
the checker validates against the rule head via check_rule_produces_equality.
But fv_can is built with *deduped* children, which can extract to a canonicalized
shape (a subterm rewritten elsewhere, e.g. matrix's nrows(NamedMat B) -> NamedDim
m) that no longer matches the syntactic rule head -> 'rule head doesn't produce
claimed equality'.

Per the plan, fv_can's proof should be the reflexive Trans(Sym(e-to-e'), e-to-e')
derived from the Congr chain, not a @rule -- so it is exempt from the rule-head
check and serves as the view's 'eclass = f(children)' proof. The natural term
keeps its @rule (its children are the as-matched forms, which do match the head).

Fixes matrix/naturals/fibonacci proof-testing; 'rule head' failures 14 -> 2
(remaining: luminal complex rule + pre-existing @p1 custom-merge).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…erms

Root cause of the proof-mode canonicalization failures on deeply-nested rules
(e.g. matrix, and the constructor parts of luminal/hardboiled_conv1d):

instrument_action's Let case bound the let variable to the built term's deduped
e-class var but did NOT carry the term's nat_conn entry onto the let name. So a
rule like `(let new-e (Bop ...)) (union e new-e)` looked up nat_conn["new-e"],
missed, and `union` fell into the no-connector branch: a bare @rule over
ordering-max/min(e, new-e) whose endpoint extracts the *deduped* (canonicalized)
shape, while process_actions builds the syntactic shape -> "rule head doesn't
produce claimed equality". Fix: propagate nat_conn from the result var to the
let-bound name.

Also keep the natural node *unseeded*: always mint a separate canonical-children
node to seed the view (never collapse to seeding fv_nat), so the natural is never
pulled into the view's congruence :merge (native UnionId) and stays as-built.
Otherwise a birewrite re-keying the view to a differently-shaped partner
(IntImm32 x <-> IntImm 32 x) natively merges the natural into the partner and its
@rule endpoint extracts the wrong shape.

Result: matrix, naturals, fib and all other constructor-based proofs prove
cleanly. Remaining: luminal still fails on its `vec-of` container (containers are
built with deduped elements and their @rule extracts the canonical shape rather
than threading the natural like constructors -- a separate follow-up), and the
pre-existing @p1 custom-merge bug is unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The connector-threading fix changed the instrumented proof structure, so all
proof-testing snapshots for passing constructor-based proofs are regenerated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nat_conn maps an instrumentation unit's freshly-minted vars (and let-bound
names) to their natural id + connector; it is local to a single generated
program. The connector-threading fix keys it by the user's let-variable names,
which repeat across rules, so without resetting it, stale entries leak between
rules/merges and can reference out-of-scope vars ("Unbound symbol @vn"). Clear
nat_conn at the start of each rule, merge function, and global action.

No current test changes result; this prevents cross-unit leaks (which a later
container change would otherwise surface).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes luminal's vec-of container proof failure with the container rebuild
itself, rather than an action-side connector.

A container built over deduped element ids has a term-proof whose endpoint
extracts the canonicalized (birewrite-partner) shape instead of the syntactic
one -> "rule head doesn't produce claimed equality" for a parent term. Building
over the *natural* (as-built) element ids fixes the proof, but naturals have no
union-find entry, so the container rebuild can't canonicalize them and the
computation breaks.

Fix: add a per-eq-sort auxiliary union-find `@UF-Aux-<Sort>` mapping a natural
id to its canonical dedup id plus the connector proof `natural = canonical`.
The container is built over natural element ids (so the term-proof extracts the
syntactic shape); at build time each element's `natural -> (canonical,
connector)` edge is written to `UF-Aux`. The Rust container rebuild
(`rebuild_container_value_rec` / `rebuild_container_proof_rec`) now consults
`UF-Aux` in addition to the main `UF`, chaining `natural ->(UF-Aux)-> canonical
->(UF)-> leader` and composing the connector with the UF proof via `Trans`. So
the rebuild canonicalizes the elements and proves it with the existing
machinery. `UF-Aux` is written only for container elements. Deterministic table
name (`@UF-Aux-<Sort>`) so the rebuild needs no extra plumbing.

Natural-element building is applied to `vec-of` only: it makes elements change
during rebuild, which triggers the rebuild's per-element `@Congr` fold, and that
fold is positional (sound only for ordered containers). Sets/maps keep the
deduped path; an order-independent rebuild proof for a birewrite-canonicalized
unordered-container element is a follow-up (no current test needs it).

Result: luminal (both variants) and all container tests prove cleanly, with no
proof-snapshot changes; only the pre-existing @p1 custom-merge failures remain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
oflatt and others added 28 commits July 31, 2026 22:20
`reflexivize_premise` built `Trans(Sym(p), p)` with bare `id_to_proof`
pushes while `proof_head` builds the same thing through
`push_shared_proof`, so the two rows of one firing could reflexivize a
premise to different ids — and the premise vector is both the head-walk
memo's key and a rule proof's sharing key.

`ProofAlgebra::reflexive` is that composition, over the shared builders.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`HeadProof::GlobalValue` was the one proof a head's walk filled with
`None` and the one no encoder site named, so every head containing a
`set` carried a padding column. A `set` claims its row and nothing else.

`HeadPosition::Set` stays distinct from `Call`: the layout assertion is
what catches a `set` lowered where the walk expects a call.

Encoding change: a numbered column after a `set` shifts down by one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rebuild skeleton was written twice — once where the encoder builds
the row, once in the test that unpacks it — so the test round-tripped
its own copy of the shape rather than the one the encoder writes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Skeleton` and `Composition` were variant-for-variant identical; the only
difference was what a leaf holds, and `Composition::pack` is the map
between them. One `ProofTree<L>` says that, and gives `Composition` the
`sym`/`trans`/`congr` builders it lacked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every constructor's arity and child positions were written twice, in
`parse_proof_inner` and again in `nested_proofs` with a different
scrutinee. A constructor added to one and not the other left only a
`debug_assert`: in release, a deep chain through it overflows the stack
instead of reporting anything.

One table says how a constructor is read, and both derive from it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- `RawProofStore::proof_to_term` was written on every parse and read
  nowhere; its doc called the pair a bidirectional map, which it is not.
- `compute_rule_substitution`'s length check is unreachable: its one
  caller asserts the premises cover the body and then zips, which
  truncates to exactly that length.
- `unify_fact` has one caller, in the same `impl`.
- `is_custom_func_fact`'s reversed `Eq(Var, Call)` arm cannot match:
  proof normal form always writes the call on the left.
- `stage_batch`'s row count is always positive — `run_instrs` returns on
  an empty mask and neither `Insert` nor `Remove` clears it — so the
  count, the return, and the branch on it do nothing.
- `source_at` and `row_sources` re-matched `ValueSource` and
  `QueryEntry`, which `mask.rs` already does; both now live there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The changelog entry described the encoding's internals -- which premises ride in
which row, what a rebuild packs. None of that is visible to someone using egglog:
the proofs are identical. What changed for them is that proof mode is faster and
smaller.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`site` and `head_chain` lived on `ProofInstrumentor` and were scoped by
assignment: set on entering a head, cleared on leaving it, saved and restored
around a `change` argument. A missed reset leaked one rule's numbering into
the next.

They are now one `Head`, threaded through the action instrumentation, so the
borrow is the scope. Every block that lowers actions makes its own — a rule
head, a top-level action, a merge body, an `extract` expression — and the two
that used to run on whatever the last head left behind now say so. The chain
loses its `Option`: there is no longer anywhere without a head to hold it.

The emission state stays on the instrumentor. `reflexive`, `deferred` and
`sealed` are filled by `instrument_facts` and read by the head that follows,
so they outlive a head; all four, `packed_decls` included, are reached through
`mint`, which every declaration and rebuild rule also calls.

Threading the head pushed four functions past the argument limit, so each
gives one up: a `union` claims its own columns, a composed union edge orders
its own endpoints, a term looks up its own e-class sort, and a construct-into
guest is bound by its caller like any other `let`.

The encoding is byte-identical over all 109 encodable tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seventeen functions took some of `stmts`, `head` and `justification`, and
fourteen took all three. Those three are one thing — where a row goes, which
head it belongs to, and why it holds — so they are now one `Emit`, and the
functions that lower an action take that instead. Nothing in the file passes
more than five arguments now; six functions used to pass seven.

Two of `Emit`'s methods say what the extra arguments used to. `justified_by`
gives the same place a different reason: a numbered column of the same rule, or
a proof composed on the spot, so a composed union edge says "fiat" once at the
top instead of at each of its four calls. `composing` runs a nested position the
head concludes nothing about.

`scope` stays out of it. It is the binding environment an expression reads, not
a place to write; only three of these functions take one, and `instrument_action`
needs it `&mut` where the others do not.

Two of the functions were long not for want of a struct but because they had a
second way out: `add_term_and_view` and `instrument_action` returned a
`Vec<String>` that every caller immediately extended onto the buffer it had
already handed them. They write to the one buffer now.

Threading the head cost four arguments last commit, and all four still stand
without the pressure. Three positions are claimed by the function that knows
which position it is — a `union`, a construct-into guest, a call — and only
`add_term_and_view`, shared by four of them, is told; a composed union edge
orders endpoints it already holds both of; `term_sort` names the lookup three
constructor builders were spelling out, and now a fourth; and a guest binds its
`let` in the caller, like the ordinary `Let` arm beside it.

The encoding is byte-identical over all 109 encodable tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An index is probed rather than scanned, so the value it is looked up by has to
be bound elsewhere in the query. The check treated every other index atom as
binding nothing, which also rejects the case where one index's *row columns*
bind the value a second index is probed by -- a query core-relations' own
occurrence-var check accepts.

Only the leading argument, the value an atom is itself probed by, is excluded
now; two index atoms naming each other there would each look bound with neither
reachable. Tested both ways: restoring the old exclusion fails the accepting
case and leaves the rejecting one passing.

Reported by Copilot on saulshanabrook#39.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An index atom is the only body atom the DD backend did not count as one, so
a rule whose body is nothing else was partitioned as atom-less: fired once
from an empty environment, its head reaching for variables the join never
bound. `plan_join` lowers that body fine, so the rule failed with "unbound
term in rule head" rather than producing its match.

A declared index registers a function type but no backend table, so a
command that looks one up by name found nothing where it expected a table:
`(input <index> …)` unwrapped a `None`, and an index over another index
reached the missing entry when a rule first queried it. Both now report the
index rather than panicking.

Also: bound an occurrence index's columns by the function's own schema
rather than the backing table's, which includes the timestamp and subsume
columns; validate an index atom's terms as the other body atoms are; walk
only the set bits of an occurrence column mask; and say in the backend SPI
that an index atom carries the index relation's unit output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two statements per node, and two term nodes per `t = t`, were most of what
the encoder emitted. The top-level block for `(Add (Num 1) (Num 2))` was 44
statements for three constructor applications; it is now 25.

A term/proof/AST node is written by `(let v (mint-<Relation>! args…))`, one
call that mints the id and asserts the row, in place of a `get-fresh!`
binding and a `set`. Every relation whose last input is a minted id — the
term relations, the `@Ast<Sort>` wrappers, every proof-node relation — gets
one, registered where its declaration is typechecked, beside the view's
`set-if-empty`, so the desugared program still re-parses. A per-relation
family rather than one generic op: the relation is then in the signature, so
the call typechecks and resolves like any other primitive, with no table
name to decode at run time. Backends service it through
`Backend::register_mint_row`.

A merge body that mints must still declare the write, or the strata that
order `merge_all` can put the minting relation in the same wave as the
merging one and leave it out of the read-only view. The bridge records the
table each row-inserting external function writes and `MergeFn::fill_deps`
declares it, which is the dependency the `set` used to carry.

A reflexive `Fiat` now names one `@Ast` node on both sides instead of
minting two over the same value. Nothing downstream changes: the two sides
were already extracted to the same hash-consed term, so `RawProof::Fiat`,
proof conversion and the checker all see what they saw before.

Encoding every `egglog/tests` program in proof mode: 199,408 statements to
153,100 (-23%), the same 109 files encoding and no file growing. Rows per
firing are unchanged (2 for a flat rewrite, 6 nested, 1 per view rebuild, 1
per merge collision); `(Add (Num 1) (Num 2))` keeps its 4 proof rows and
goes from 6 `@Ast` rows to 3.

The one snapshot that moves is `doc_example_add_function1`, which dumps the
encoded program; its diff is two fused mints and nothing else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The proof system had no way to get from a proof about `f(a, b, c)` to a
proof about `a`, so the only way to say a term had been built was to look
its reflexive proof up in `@<Sort>Proof`. Add the projection rule the
mechanized system has: from `t = f(..., ci, ...)` and the index of `ci`,
conclude `ci = ci`.

It is stated at the equality level rather than as a separate "is built"
judgement, because in egglog `t = t` already is that judgement --
reflexivity is not assumed, so a proof of it is exactly a witness that
`t` was built. That collapses the mechanized system's projection and
reflexivity rules into one.

The encoder declares `@Proj` but writes none: every reflexive anchor it
needs it already holds, and deriving one instead costs a minted row
where the table costs a lookup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`@<Sort>Proof` held a `t = t` proof for every term and every container —
11,972 writes over the corpus, 15% of every statement the encoder emits —
so that anywhere a reflexive anchor was wanted it could be looked up. Every
one of those places already has a row proof in scope whose right-hand side
mentions the term, so the anchor is a projection out of it: `Proj` at a
child position, `Trans`/`Sym` for a constructor view's e-class, and a new
raw-only `ProjAll` — `CongrAll`'s projection counterpart, desugared to
`Proj` during conversion — for a container element, whose position in the
term form is only known once the term is in hand.

Which row anchors a rule body's variable is only known once the whole body
is walked, so the body collects what each view atom offers and binds the
anchors at the end; a value a body primitive reads out of a container has
no anchor at all and becomes a side condition the checker re-evaluates,
alongside the container-producing primitives that already were. Conversion
now binds a side condition's output by the same evaluation, so a rule head
may use it.

With nothing left reading a term's own conclusion, a rule head writes no
row for it either. The corpus goes from 77,101 statements to 58,493; a
flat rewrite writes 1 proof row where it wrote 2, the nested head in
proof_encoding.md 3 where it wrote 6, and a view rebuild one packed row
plus one anchor per canonicalized column where it wrote one packed row.
`web-demo/math`, `herbie` and `typeinfer` run in 0.90x, 0.89x and 0.92x
their wall time.

The proofs that move are anchors changing provenance, never a proof
changing shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`uf_canon_proof` answers a rebuild's "did this column move?" with the
column's `@UF` row proof, or, on the miss that says it did not move, with
the column's reflexive anchor. The caller minted that anchor eagerly and
passed it in, so every firing wrote one anchor row per canonicalized
column and threw away the ones whose column had moved — four rows per
firing for a binary constructor where one row is the point.

The primitive now takes what the projection needs, the row proof and the
column's position, and mints `Proj(row_proof, position)` itself on the
miss. Backends service the composed op through the new
`Backend::register_view_column_read_or_mint`, so it stays a view-column
read the backend answers against its own storage rather than a registry
primitive the Differential Dataflow backend cannot run.

A constructor view's e-class is the row proof's left-hand side rather
than one of its children, so it takes the projection position -1, which
parsing reads back as the reflexivization `Trans(p, Sym(p))` the rebuild
used to spell into a `@Packed_1` row. Same proof, same node, written one
call later. The container rebuild keeps its eager anchor: it composes
from that anchor on every call, so there is no miss to defer to.

A firing of a binary constructor's rebuild goes from 4 proof rows to 1
plus one anchor per column that stayed put, and the corpus from 58,493
statements to 56,263. `web-demo/math`, `herbie` and `typeinfer` run in
0.97x, 0.99x and 0.95x their wall time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`(= (vec-get ys 0) (vec-get ys 1))` is settled by the primitives alone, but
the side-condition gate demanded a literal variable on one side, so the fact
took the premise path and asked for a reflexive anchor on each element. An
anchor is a projection out of a row proof the body reads, so a container the
body *built* — `(= ys (vec-of ...))` — anchors nothing, and the request
panicked. Its variable-shaped sibling `(= e (vec-get ys 0))` was already a
side condition and already worked.

Admit primitive result against primitive result too. The fact routes to
`check_side_condition`, whose both-sides-determined branch re-evaluates each
side and compares — which is what verified the fact before as well, since a
premise composed from two reflexive anchors reduces to a reflexive proof and
`check_fact_matches_proposition` then requires both sides to agree with it.
The encoder emits the fact as-is with the `Eval` marker and requests no
anchor.

A built term against an element read, `(= (Add a b) (vec-get v 1))`, stays
out: it asserts an e-graph equality re-evaluation cannot settle, and keeps
its real premise.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tayed put"

This reverts commit d3fb7e1.

Deferring an anchor to the `@UF_<Sort>` miss cut the corpus by 3.8%, but
moved neither wall time (0.96-0.99x, within noise) nor peak RSS (49/49,
65/61, 36/36 MiB on web-demo/math, herbie, typeinfer). It cost a new
backend SPI method threaded through egglog-bridge and the DD backend's
action and merge paths, and a sentinel projection position standing for
a reflexivization. A smaller emitted program is not on its own worth
that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the checker

Anchoring a term by projecting it out of a row proof left two body shapes with
no row to project from, and the gate the checker uses to skip a premise was
widened twice to cover them: an eq-sort value read out of a container the query
itself built, and one a primitive produced without being handed a container at
all. The checker is the trusted base, so it should not move to accommodate the
encoder. Restore it to the projection rule and nothing else, and reject those
two shapes up front instead.

A container's reflexive proof cannot be derived either: `@Congr`/`@CongrAll`
rewrite a child of a term some proof already mentions, so neither can introduce
the container's term in the first place, and `@Fiat` is reserved for literals
and value-constructor terms. There is no introduction rule for a term the
database never held, which is the point -- a value the query computed is one
proofs have nothing to say about.

Two new `ProofEncodingUnsupportedReason`s say so, alongside the query-built
container an action may not carry. `container-proofs.egg`'s one affected rule
moves to `container-query-built-read.egg`, which is expected-unsupported, so the
rest of the file keeps encoding under proofs.

A body's equalities now join the variables they relate into one anchor class, so
`(= e (vec-get v 0)) (= f (vec-get v 1)) (= e f)` over a matched container is
anchored through the reads rather than asking after `f` alone. Those premises go
back to a real `@ProjAll` off the container's row where the widened gate had
left them an `@Eval` marker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two rejections added with the projected anchors fired on the shape of a body
fact: any eq-sort value read out of a container the query built, and any eq-sort
value a primitive produced with no container to read it out of. Both are broader
than the encoder is. A fact's premise composes `Sym(left)` with `right` and drops
whichever side proves `t = t`, so an equality reads its right-hand anchor exactly
when its left-hand proof is itself reflexive. A view atom's argument, the
right-hand side of a term the query builds, and a value another atom anchors
after all all encode fine today, and were being turned away.

Scan the body for what the encoder's `BodyAnchors` would collect -- which values
a view row reaches, which equalities alias them, which reads came out of which
container -- and reject only a premise that reads an anchor no view row supplies.
Which value that is decides the reason, so both keep their meaning: a container
read names the container, and one with no container names the primitive.

Note that it is the premise, not the actions, that reads such an anchor: the
premise of the fact binding the value *is* its reflexive proof, so removing the
value's use from the head changes nothing. The rejection tests say so now, and
`container-query-built-read.egg` stays unsupported for that reason.

A head that mints no row reads no premise at all, so those rules would encode
too. Proof support does not model which actions read the rule proof and rejects
them, which is the safe side of the invariant the encoder asserts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every encoded constructor carried a deferred-delete and a deferred-subsume
relation, a rule for each, and one marker-rebuild rule per eq-sort column,
whether or not the program ever deleted or subsumed. Only 46 of 696 corpus
constructors ever do.

The delete indirection bought nothing. The backend already stages a removal to
the end of the batch, and a table's commit applies removals ahead of insertions,
so lowering `delete` straight to the view row reproduces the uninstrumented
meaning. The marker instead delayed the removal to the next maintenance pass,
which deleted a row a rule inserted in the same batch, missed a row already
subsumed, and missed a row the intervening rebuild had re-keyed.

Subsumption still needs its marker, because rebuilding re-keys a view row by
inserting it afresh and the new row carries no subsumed bit. Its scaffolding is
now emitted ahead of the first command that subsumes the constructor. Commands
arrive one at a time, so a pass over the program cannot find them; the memo
lives on the e-graph, which `push` clones and `pop` restores along with the
declarations it tracks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The container rebuild minted an element's `@Ast` node and its `@ProjAll`
anchor before knowing whether the element changed, then discarded both —
along with the whole recursive proof standing on them — whenever it had
not. Ask the value-level rebuild first, as the eq-sort branch beside it
already does, and mint nothing for an element that stays put.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This reverts commit cc807d5.

Deferring a nested container element's anchor until the element is known
to have moved is correct, and on a purpose-built (Vec (Vec E)) probe it
fires. On the corpus it cannot: @ProjAll holds zero rows on both
eggcc-2mm and hardboiled_conv1d_32, so the nested-container branch is
never entered. Wall time and peak RSS are unchanged within noise.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A `@Fiat` named its two endpoints by wrapping each in an `@Ast<Sort>` row,
whose only job was to erase the endpoint's sort so one relation could take
both. Specializing the relation on the sort does the same erasure at the
declaration instead: `@Fiat_<Sort> (S S @proof)` names the values directly,
so a reflexive fiat costs one row rather than two.

The declaration goes with the first command needing it, like `@Packed_<k>`
and `@to_subsume_<Ctor>`, and the name is derived from one prefix so a
re-parsed program recovers it from the `@Proof` sort's `:internal-proof-names`
without having encoded anything.

eggcc-2mm: 2.081M -> 2.005M rows (-3.6%), 17 `@Fiat_<Sort>` declarations
against 120 `@Ast<Sort>` tables of which those same 17 ever held a row;
forms 2722 -> 2738, RSS 397 -> 388 MiB, wall unchanged.
hardboiled_conv1d_32: 345.8k -> 328.4k rows (-5.0%), forms 1035 -> 1049.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`@ProjAll` named the projected child through an `@Ast<Sort>` row, the last
reader of the wrapper. Specializing it the same way the fiat justification
now is — `@ProjAll_<Sort> (@proof S @proof)` — names the child directly, and
the `@Ast` sort, its per-sort wrappers, their mint primitives, the
`:internal-ast-func` annotation and `unwrap_ast` all go with it.

A container sort's projection is declared with the sort rather than on first
use: the rebuild primitive mints one for a nested container without going
through any statement the encoder wrote. Proof mode is now read off the
`@Proof` sort being declared, which is what `:internal-ast-func` was standing
in for at `register_uf_canon`.

eggcc-2mm: 2722 -> 2617 top-level forms across the two commits (-3.9%), of
which 121 go here (120 wrappers plus the sort). hardboiled_conv1d_32:
1035 -> 1003. Rows and wall unchanged here — the wrapper's rows went with the
fiat change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A body fact over base values — a guard like `(> n 0)`, or the `(= len n)` a
custom function's output leaves behind — has a reflexive fiat for its premise,
whose term is a function of the fact and the bindings the earlier facts made.
Conversion can re-evaluate it, so the encoding writes neither the fiat row nor
the premise column that would name it, and the firing's `@Rule_<k>` is one
column narrower.

One gate over the body decides which facts these are, shared by the encoder
and proof conversion so the two cannot drift, and taken in body order because a
fact is only recomputable once one of its sides reads nothing but already-bound
variables. Conversion therefore walks the body once, interleaving premise
conversion with the substitution it feeds, instead of converting every premise
up front. The checker is untouched: it already pins these propositions, since
it re-evaluates each fact against the full substitution to match it.

eggcc-2mm: 2.004M -> 1.930M rows (-3.7%), 75.1k `@Fiat_<Sort>` rows down to
567. hardboiled_conv1d_32: 329k -> 308k (-6.4%), 18.7k down to 980.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sweep the crate for prose describing machinery this branch deleted: the
per-sort reflexive proof tables, the AST wrapper sorts, and the deferred
deletion markers. Fold the dangling doc block the removed
`compute_rule_substitution` left above `rule_premises` into that function's
own contract, and trim the doc comments this branch added down to what a
caller needs.

Clear `unanchored` with the rest of the held-back state, so a body whose
anchor goes unread does not keep the deferred-lookup fast path off for
every later rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oflatt-claude

Copy link
Copy Markdown
Owner Author

Superseded by saulshanabrook#48, opened against upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants