Store a function's signature once, and expose it to primitives

.claude/skills/tidy-diff-docs/SKILL.md

@@ -45,6 +45,17 @@ Two guardrails:
   need not repeat the shape; if a parameter's meaning is obvious from its name
   and type, do not spell it out again.
 
+## Also fix in the diff
+
+- **Inline full paths for types.** Import the type and use the short name.
+  Write `use crate::util::HashMap;` and then `HashMap<String, Foo>`, not
+  `crate::util::HashMap<String, Foo>` in a field or a signature. A qualified
+  path inline makes the type harder to scan and hides the dependency from the
+  module's import list. Two exemptions: intra-doc links (``[`crate::Foo`]``)
+  need the path to resolve, and a path may disambiguate two same-named types in
+  scope. A qualified one-off *call* such as `std::mem::take(..)` is idiomatic
+  and not worth an import.
+
 ## Keep these
 
 - What the item does and the shape of what it returns.

CHANGELOG.md

@@ -5,6 +5,7 @@
 - Fix a crash where bounded table scans with column constraints could yield stale (deleted) rows, leaking the stale value marker into primitives and panicking with an out-of-bounds intern-table read (e.g. `index out of bounds: the len is 0 but the index is 2147483647`).
 - Fix a build failure when egglog is compiled without default features (as a library dependency). The `egglog-add-primitive` proc macro parses full Rust expressions and now declares `syn`'s `full` feature directly, instead of relying on another crate (`clap_derive`, via the `bin` feature) to unify it onto our `syn`. This surfaced when `clap_derive` moved to `syn` 3.x. A CI job now builds `-p egglog --no-default-features` to catch regressions.
 - Convert many reachable `panic!`/`unwrap`/`expect`/`todo!` sites into recoverable errors, so malformed or edge-case programs report an error instead of aborting the process. Examples now returning `Error`/`TypeError`/`ParseError`/`ProveExistsError`: malformed sort-constructor declarations like `(sort S (Vec))` or `(sort S (UnstableFn))`; negative `extract` variant counts; duplicate rule names; `unstable-fn` referencing an unknown, non-literal, or mis-typed target; `(fail ...)` wrapping `include` or an empty expansion; subsuming a non-call rewrite; running `prove`/`prove-exists` without proofs enabled; and missing/unreadable files for `input`, `print-function`, `print-overall-statistics`, and the CLI. Several primitives became partial (returning no result instead of panicking) on out-of-range input: `vec-set`/`vec-remove` indices, `multiset-pick` on an empty multiset, count overflow in `multiset` operations, and the numeric primitives `bigint <<`/`>>`, `bigrat`, and `log2`. A few scheduler edge cases (unknown ruleset, rules with no free variables) no longer panic; variable-free rules now correctly apply their actions when scheduled. Primitive resolution now returns `TypeError::AmbiguousPrimitive`/`TypeError::UnresolvedPrimitive` instead of panicking when duplicate same-signature registrations are indistinguishable or nothing resolves; both direct calls and `unstable-fn` primitive targets report the same variants. `step_rules_with_scheduler` now restores its `rulesets`/`schedulers` on every fallible path, so an error during scheduled rule compilation no longer leaves the `EGraph` in a corrupted state.
+- **Breaking:** a function's signature is stored once, shared between `TypeInfo` and `Function` instead of copied into both. `TypeInfo::get_func_type` returns `Option<Arc<FuncType>>`, `Function::schema()` becomes `Function::func_type() -> &FuncType`, and `ResolvedSchema` is removed (its `get_by_pos` moves to `FuncType`).
 - **Breaking:** `EGraph::print_function` now takes its output sink as `Option<(File, PathBuf)>` plus a `Span`, so write failures return `Error::IoError` instead of panicking.
 - Speed up query evaluation by building on-the-fly per-subset column indexes as sorted arrays (`SortedColumnIndex`) instead of hash maps. These indexes are typically iterated once and probed a bounded number of times over high-cardinality columns, so skipping hash-table construction is a large win (e.g. ~33% faster on the `gemma` benchmark).
 - Share trie roots (and their cached sub-indexes and child nodes) across query plans within a single `run_rule_set` instead of rebuilding a fresh trie per plan. Plans that scan the same table under the same header (fast) constraints reuse one root, so on-the-fly per-subset index builds happen once rather than per plan; only roots that more than one plan uses are shared, so workloads that would not benefit keep the per-plan behavior. Large speedups on transformer workloads (e.g. ~15% faster on `whisper`, ~12% on `gemma`, ~8% on `qwen3_moe`).
@@ -27,7 +28,7 @@
 - **Typed primitive surface for seminaive safety (#772).** Custom primitives now pick one of `PurePrim` / `ReadPrim` / `WritePrim` / `FullPrim` based on what the body needs, and register via the matching `add_*_primitive`. Rust enforces capability bounds via the state wrapper passed to the body; the egglog typechecker enforces context bounds. See the `egglog::exec_state` module docs and the `*Prim` trait docs for the full picture. Migration: `rust_rule` callbacks now take `&mut WriteState` (replacing `RustRuleContext`); a new `rust_rule_full` gives action callbacks read access. Higher-order primitives over `unstable-fn` values dispatch via `state.apply_function(&fc, args)`.
 - Expose `Read::table_size(name)` and `Read::table_sizes()` so read-capable primitives can inspect row counts without raw execution-state access, while avoiding an all-table scan when only one table is needed.
 - **`:naive` and `:unsafe-seminaive` rule options** (mutually exclusive). Both compile a rule under the permissive `Read`/`Full` contexts so its RHS can read the database (read-primitives and function-table lookups). `:naive` matches the whole database every iteration; `:unsafe-seminaive` keeps seminaive (delta) matching, which is faster but **unsafe** — an RHS read observes the database mid-iteration, so results can depend on evaluation order. `:unsafe-seminaive` is rejected by the term/proof encoding.
-- **Name-indexed e-graph access from primitives and `rust_rule` callbacks (#745, #751).** New `Read` / `Write` capability traits on the state wrappers let primitive bodies and rule callbacks read/write tables by name (`fs.lookup`, `fs.set`, `fs.add`, `fs.union`, `fs.function_entries`, `fs.constructor_enodes`, etc.) instead of through raw `FunctionId` + `&[Value]`; `EGraph::update(|fs| ...)` gives the same surface outside a rule, and `EGraph::function_entries` / `EGraph::constructor_enodes` expose the table scans directly at the top level. Misuse (wrong subtype, wrong arity, unknown table) surfaces as `Error::ApiError`.
+- **Name-indexed e-graph access from primitives and `rust_rule` callbacks (#745, #751).** New `Read` / `Write` capability traits on the state wrappers let primitive bodies and rule callbacks read/write tables by name (`fs.lookup`, `fs.set`, `fs.add`, `fs.union`, `fs.function_entries`, `fs.constructor_enodes`, etc.) instead of through raw `FunctionId` + `&[Value]`; `EGraph::update(|fs| ...)` gives the same surface outside a rule, and `EGraph::function_entries` / `EGraph::constructor_enodes` expose the table scans directly at the top level. Misuse (wrong subtype, wrong arity, unknown table) surfaces as `Error::ApiError`. Also `Read::enodes_for_eclass` (a constructor's rows by output e-class, through the backend's column index rather than a scan), `Read::constructor_schema` / `Read::function_schema` / `Read::table_subtype` (a table's declared signature and subtype, which a primitive body cannot get from `TypeInfo`), and `Core::rebuild_container` (remap a container value's contents and intern the result, for container sorts whose Rust type the caller cannot name). Together these let an out-of-tree primitive walk and rebuild a sub-e-graph — see `unstable-subst` in `egglog-experimental`.
 - **Container support in the term/proof encoding.** Programs using container sorts (`Vec`, `Set`, `Map`, `MultiSet`, `Pair`) now work under the term/proof encoding (previously rejected), including containers read (`vec-get`, `map-get`, …) or constructed (`vec-of`, `set-of`, …) in a rule body (`set-get` excepted: it indexes an internal runtime order that proofs cannot reproduce). A container built in the body is a *side condition* with no carryable proof: it is marked with an `Eval` proof step and re-evaluated against the typed rule when checked, so it can be read or matched in the query but not carried into an action (that is rejected). Two user-visible extraction changes: container terms extract in a deterministic, reproducible order rather than value-id order, and maps extract in a flat `(map-of k0 v0 …)` form (new `map-of` constructor) instead of nested `map-insert`s.
 
 ## [2.0.0] - 2026-02-11

core-relations/src/action/mod.rs

@@ -12,18 +12,20 @@ use std::{
 
 use crate::{
     common::HashMap,
-    free_join::{invoke_batch, invoke_batch_assign},
+    free_join::{get_column_index_from_tableinfo, invoke_batch, invoke_batch_assign},
     numeric_id::{DenseIdMap, NumericId},
 };
 use egglog_concurrency::NotificationList;
 use smallvec::SmallVec;
 
 use crate::{
-    BaseValues, ContainerValues, ExternalFunctionId, WrappedTable,
+    BaseValues, ContainerValues, ExternalFunctionId, Offset, WrappedTable,
     common::Value,
     free_join::{CounterId, Counters, ExternalFunctions, TableId, TableInfo, Variable},
+    offsets::Subset,
     pool::{Clear, Pooled, with_pool_set},
-    table_spec::{ColumnId, MutationBuffer},
+    row_buffer::TaggedRowBuffer,
+    table_spec::{ColumnId, Constraint, MutationBuffer},
 };
 
 use self::mask::{Mask, MaskIter, ValueSource};
@@ -483,6 +485,66 @@ impl<'a> ExecutionState<'a> {
         &self.db.table_info[table].table
     }
 
+    /// Iterate over visible rows in `table` whose `col` equals `value`.
+    ///
+    /// Cacheable columns use the table's lazy column index; uncacheable columns
+    /// fall back to scanning with the equality constraint. The callback
+    /// receives each matching row as a full value slice.
+    pub fn for_each_matching_col(
+        &self,
+        table: TableId,
+        col: ColumnId,
+        value: Value,
+        mut f: impl FnMut(&[Value]),
+    ) {
+        let table_info = &self.db.table_info[table];
+        let constraint = Constraint::EqConst { col, val: value };
+        let (mut subset, _fast, mut slow) = table_info
+            .table
+            .split_fast_slow(std::slice::from_ref(&constraint));
+
+        debug_assert!(slow.iter().all(|c| matches!(c, Constraint::EqConst { .. })));
+
+        if !*table_info
+            .spec
+            .uncacheable_columns
+            .get(col)
+            .unwrap_or(&false)
+        {
+            let index = get_column_index_from_tableinfo(table_info, col);
+            match index.get().unwrap().get_subset(&value) {
+                Some(s) => {
+                    with_pool_set(|ps| subset.intersect(s, &ps.get_pool()));
+                }
+                None => {
+                    subset = Subset::empty();
+                }
+            }
+            slow.clear();
+        }
+
+        let imp = &table_info.table;
+        let cols: SmallVec<[_; 8]> = (0..imp.spec().arity()).map(ColumnId::from_usize).collect();
+        let mut cur = Offset::new(0);
+        let mut buf = TaggedRowBuffer::new_inline(imp.spec().arity());
+
+        macro_rules! drain_buf {
+            ($buf:expr) => {
+                for (_, row) in $buf.non_stale() {
+                    f(row);
+                }
+                $buf.clear();
+            };
+        }
+
+        while let Some(next) = imp.scan_project(subset.as_ref(), &cols, cur, 1024, &slow, &mut buf)
+        {
+            drain_buf!(buf);
+            cur = next;
+        }
+        drain_buf!(buf);
+    }
+
     /// Get the human-readable name for a table, if one exists.
     pub fn table_name(&self, table: TableId) -> Option<&'a str> {
         self.db.table_info[table].name()

core-relations/src/free_join/mod.rs

@@ -863,7 +863,10 @@ fn get_index_from_tableinfo(table_info: &TableInfo, cols: &[ColumnId]) -> HashIn
 /// The core logic behind getting and updating a column index.
 ///
 /// This is the single-column analog to [`get_index_from_tableinfo`].
-fn get_column_index_from_tableinfo(table_info: &TableInfo, col: ColumnId) -> HashColumnIndex {
+pub(crate) fn get_column_index_from_tableinfo(
+    table_info: &TableInfo,
+    col: ColumnId,
+) -> HashColumnIndex {
     let index: Arc<_> = table_info.column_indexes.get_or_insert(col, || {
         Arc::new(ResettableOnceLock::new(Index::new(
             vec![col],

egglog-bridge/src/lib.rs

@@ -1454,6 +1454,28 @@ impl TableAction {
         drain_buf!(buf);
     }
 
+    /// Iterate over rows whose output column is `value`.
+    ///
+    /// This reaches the table through an [`ExecutionState`], so read-capable
+    /// state wrappers can use the same lazy column indexes as direct backend
+    /// callers without exposing lower-level scan buffers or projection details.
+    pub fn for_each_output_value(
+        &self,
+        state: &ExecutionState,
+        value: Value,
+        mut f: impl FnMut(ScanEntry<'_>),
+    ) {
+        let schema_math = self.table_math;
+        let output_col = ColumnId::from_usize(self.input_arity());
+        state.for_each_matching_col(self.table, output_col, value, |row| {
+            let subsumed = schema_math.subsume && row[schema_math.subsume_col()] == SUBSUMED;
+            f(ScanEntry {
+                vals: &row[0..schema_math.func_cols],
+                subsumed,
+            });
+        });
+    }
+
     /// Look up a row, inserting the configured default value if absent.
     /// For constructor tables this mints a fresh eclass ID; for custom
     /// functions (no default) this behaves identically to

src/core.rs

@@ -158,7 +158,7 @@ impl ResolvedCall {
             let expected = ty.input.iter().map(|s| s.name());
             let actual = types.iter().map(|s| s.name());
             if expected.eq(actual) {
-                return Some(ResolvedCall::Func(ty.clone()));
+                return Some(ResolvedCall::Func(ty.as_ref().clone()));
             }
         }
         None
@@ -175,7 +175,7 @@ impl ResolvedCall {
             let expected = ty.input.iter().chain(once(&ty.output)).map(|s| s.name());
             let actual = types.iter().map(|s| s.name());
             if expected.eq(actual) {
-                return Ok(ResolvedCall::Func(ty.clone()));
+                return Ok(ResolvedCall::Func(ty.as_ref().clone()));
             }
         }