Winograd streaming forward pass: 5.6-5.9x throughput, portable and safe

[ealmloff]

Summary

Rebuilds the forward pass as a streaming, length-aware Winograd pipeline for a ~2x-5.9x single-inference throughput gain (see table), built entirely on fearless_simd — the crate compiles under #![forbid(unsafe_code)], with no FFI, no intrinsics, no platform cfg, no new dependencies, and the same engine on every target. The old repeated-tail tensor pipeline is deleted outright (~750 lines; layers.rs goes 951 → 81 lines).

Commit 1 — streaming engine:

• Padded input buffers — conv padding is materialized, so the kernels are branch-free with no bounds checks or edge cases in the hot loops.
• Precomputed tails (ForwardPlan, once per process) — every row at or beyond the data-dependent range of a stage equals the same row of a forward pass over an all-padding sequence; suffix max/sum tables make the global pool O(1) in the padded tail. Per call, only input-dependent rows are computed.
• Endpoint max-pooling — GELU is unimodal (single minimum at x ≈ -0.7517), so max(gelu(x_i)) over a pool group is attained at an endpoint of [min x_i, max x_i]: two GELU evaluations per pooled output, fused in registers.

Commit 2 — Winograd + assembly-driven kernel work:

• The disassembly showed the original inner loop spending 695 instructions per 144 FMAs on accumulator spills, bounds checks, and re-streaming the whole kernel per position block. After packing weights into contiguous per-16-channel groups, moving the group loop outermost, and register-blocking the accumulators, the hot loop is 22 instructions per 16 FMAs (~95% of the NEON f32 FMA ceiling).
• At the FMA ceiling, only fewer multiplies go faster: every conv stage now runs as F(4, k) Cook-Toom minimal filtering (1.9-2.8x multiply reduction). Transform kernels are generated from exact rational matrices by scripts/gen_winograd.py, with ±a interpolation points factored into shared even/odd halves. fp32 drift vs direct convolution is ~1e-5, two orders below the runtime's Padé GELU approximation error.
• The reference pipeline is replaced by a plain scalar oracle written in test code; the forward plan bootstraps its tail rows by running the Winograd stages once over an all-padding sequence.

cargo bench --bench detect, M2 Max, same machine:

Case	main	this PR	speedup
short (68 B)	134.1 µs	23.8 µs	5.6x
full window (4.6 KB)	2.277 ms	386.3 µs	5.9x

detect() reads a fixed ≤2 KB window, so per-file cost is constant in file size — a 1 MB input classifies at multiple GB/s; the bench cases are the worst case by construction (noted in BENCHMARKS.md). The short case is floor-bound by streaming the ~720 KB of transformed weights through a fixed-cost input.

Test plan

• cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings, typos
• cargo test --release: 27 tests + 10 doctests, including a fuzz test comparing the engine against a naive scalar oracle across 18 unit-sequence lengths (tolerance 1e-2 on logits) and a unit test for the pooled endpoint identity over adversarial negative mixes
• #![forbid(unsafe_code)] enforced at the crate root
• Tree-mode output on a local wasm-bindgen checkout byte-identical (95.38% accuracy, unchanged)

Generated with Devin