Training support - #430

Draft

ealmloff wants to merge 89 commits into

mainfrom

Commits on Mar 15, 2026

backprop init
ealmloff
committed
fbb24ad
transformer example
ealmloff
committed
235d246
faster
ealmloff
committed
0528e0c
nanochat
ealmloff
committed
8b6839a

Commits on Mar 17, 2026

wip
ealmloff
committed
1ba33f6

Commits on Mar 18, 2026

drawing model + coop matmul
ealmloff
committed
bbb9cd3
data augmentation
ealmloff
committed
c3b8b6d
better dataset
ealmloff
committed
28bd897
wip
ealmloff
committed
8c25d67
multible outputs
ealmloff
committed
9228712
refactor
ealmloff
committed
f3002a9

Commits on Mar 19, 2026

ui close to working
ealmloff
committed
973ffc7
ui working
ealmloff
committed
9fe42a9
tab autocomplete
ealmloff
committed
f315984

Commits on Mar 21, 2026

remove output files
ealmloff
committed
a5d8f57
remove more output files
ealmloff
committed
d6ac082
move composite ops from core to fusor
ealmloff
committed
f192867
remove pairwise
ealmloff
committed
df2e0cf
fix nanochat app drawing
ealmloff
committed
14cf293
more data
ealmloff
committed
f58241d

Commits on Mar 22, 2026

better token encoding and more dynamic synth data
ealmloff
committed
986cc19
the start of a testing framework
ealmloff
committed
38fbb20
improved fuzzing cpu mode
ealmloff
committed
4b727e9

Commits on Mar 23, 2026

more tests
ealmloff
committed
602ad86
fix fuzzing tests
ealmloff
committed
ca570c7
fix strided softmax
ealmloff
committed
3cc7595
fix clippy
ealmloff
committed
8428e87

Commits on Mar 24, 2026

move more tests into conformance
ealmloff
committed
fb16b73
more fuzzy tests
ealmloff
committed
b2df02a
more fuzzing
ealmloff
committed
bcdcca3
pull out more common ops
ealmloff
committed
1969d1e

Commits on Mar 25, 2026

expose layer_norm_last_dim_fused
ealmloff
committed
21abc15
move bench
ealmloff
committed
ef4f983
more ops
ealmloff
committed
1abdae6
fix warnings
ealmloff
committed
2079595

Commits on Jun 2, 2026

Merge remote-tracking branch 'floneum/main' into backprop
ealmloff
committed
05d3a65

Commits on Jul 1, 2026

more cleanup
ealmloff
committed
261ee9a
Merge remote-tracking branch 'floneum/main' into backprop
ealmloff
committed
39532da

Commits on Jul 2, 2026

drop demo app crates and revert incidental deltas vs main
ealmloff
committed
02a4378
gate chunked submits to training-scale graphs, drop redundant first-target tracking
ealmloff
committed
db9cb90
shrink autograd surface: drop unused ops, dedup per-rank backward rules
ealmloff
committed
5cdab21
compile broadcast and restride backwards through the graph
ealmloff
committed
efa105c
express index and embedding backwards as graph ops, defer qmatmul dequant
ealmloff
committed
9a2b6e8
add gradient tests for untested backward ops
ealmloff
committed
847389b
split autograd into module directory
ealmloff
committed
74413f7
fix clippy in autograd module
ealmloff
committed
d3be0d3
restore main's exact cargo lock
ealmloff
committed
e1b967d
support tensor-tensor comparison ops on gpu
ealmloff
committed
f3e91ec
run every autograd test on both cpu and gpu
ealmloff
committed
e467f6b
add end-to-end xor classifier training test
ealmloff
committed
6ffd801

Commits on Jul 3, 2026

default api supports backprop
ealmloff
committed
97ee879

Commits on Jul 5, 2026

optimize kernels for large batch sizes
ealmloff
committed
d1ade3f

Commits on Jul 11, 2026

more fusion improvements
ealmloff
committed
f7b580b
Optimize fusion planning and transformer execution
ealmloff
committed
af2f8d1
replace the rewrite fixpoint with an egg equality saturation optimizer
ealmloff
committed
859871e

Commits on Jul 12, 2026

simplify tensor conversion and layer forward APIs
ealmloff
committed
c95a837
cap in-flight gpu memory in the buffer pool
ealmloff
committed
07ddb6d
enable semantic egraph identity and shared materialization
ealmloff
committed
7d4fe3c
fix stale index
ealmloff
committed
e5d3e42
derive dispatch sizing from a device occupancy policy
ealmloff
committed
d696687
rebind interned payload inputs to the extraction's actual children
ealmloff
committed
8a24659
fix the 128x128 coop tile and sweep every table entry for conformance
ealmloff
committed
078a5d1
replace the coop tile ladder with a measured general cost model
ealmloff
committed
25d6c5a
fuse unary epilogues into cooperative matmul
ealmloff
committed
bf5cd31
reuse fusion plans across repeated layers by default
ealmloff
committed
2f24b2f
score split-k by the occupancy wave model and collapse CoopGemmParams into the scored tile selection
ealmloff
committed
6a06625
make graph optimization size-independent
ealmloff
committed
910ef4d
single-source qgemv dispatch geometry from the kernel's shape selection
ealmloff
committed
b4244c3
pin fallback-family selection tables with structural legality properties
ealmloff
committed
b2f87d9
add coop matmul run-to-run determinism gate and drop dead policy helpers
ealmloff
committed
9493a69
pass the selected qgemv shape through the builders and drop the coop kind payload
ealmloff
committed
5ff4346
absorb outside-consumer producers as region outputs
ealmloff
committed
4eae00e
Optimize repeated fusion planning
ealmloff
committed
a6c132e

Commits on Jul 13, 2026

Commits on Jul 18, 2026

flash attention kernel via coop store-to-tile
ealmloff
committed
e61a38c
strided rank-4 operands and gqa in the flash attention kernel
ealmloff
committed
e419487
flash attention backward kernels: lse, dq, dkv
ealmloff
committed
d6ad25d

Commits on Jul 22, 2026

dtype-generic f16 training, workgroup tile arena, measured coop tile selection, >65535-tile matmul grids
ealmloff
committed
f796bf6

Commits on Jul 24, 2026

drop unused elide import
ealmloff
committed
6c0bc03
transformer example at 20m params
ealmloff
committed
15d8a76

Commits on Jul 25, 2026

Commits on Jul 26, 2026