next priorities

[j143]

Core principles (one-liners)

• Make optimizer purely analytical: never materialize data during pattern matching.
• Separate concerns: Analysis (trace + match) → IR rewrite (fuse) → Execution (apply kernels).
• Define small, stable contracts for each layer (Plan/Node metadata, BufferManager, Backend).
• Make fusion selection deterministic, testable, and fast (no I/O).
• Treat heavy tests and benchmarks as gated, cached jobs in CI.

Priority roadmap

1. Stop side-effects in optimizer

   • Remove any calls to node.execute() inside optimizer/analysis. Replace with metadata-only inspection APIs.

2. Create two-stage optimizer

   • analyze(plan) -> Trace + MatchResults
   • rewrite(plan, match) -> FusedPlan (new node types)
   • execute(fused_plan, backend, buffer_mgr)

3. Introduce immutable, hashable Plan representation

   • deterministic hashing for plan diffs, caching, and baseline comparisons.

4. Add a small, fast unit test surface

   • Opt tests: trace generation, pattern detection, rewrite correctness (use mocked backend).

5. Add integration smoke tests that run fused kernels on tiny matrices (CI quick job).

6. Add benchmarks & regression checks

   • Baselines stored as CI artifacts; regress only if delta > threshold.

7. Plugin backend API

   • Simple interface for kernels: (inputs, params, output_path, buffer_mgr), so swapping implementations is trivial.

8. Observability

   • Trace-level logging, cost model hooks, and per-plan flame profiles.

[j143]

Implemented in #49