Skip to content

release: v0.3.0 — gather/scatter, async Metal, fused Adam, linalg, einsum, f64, param groups (#23, #25–#30) - #31

Merged
vyncint merged 11 commits into
mainfrom
v0.3.0
Sep 3, 2026
Merged

release: v0.3.0 — gather/scatter, async Metal, fused Adam, linalg, einsum, f64, param groups (#23, #25–#30)#31
vyncint merged 11 commits into
mainfrom
v0.3.0

Conversation

@vyncint

@vyncint vyncint commented Sep 3, 2026

Copy link
Copy Markdown
Owner

0.3.0 — the "what oxmega needs" milestone, plus the pins triage

Closes #23, #25, #26, #27, #28, #29, #30.

What changed

Measured

  • Metal, oxmega device-bench (M4 Pro, 100 epochs × 3 stacked seeds, back-to-back with the published 0.2.0, three rounds): 19.4–20.1 s → 7.4–7.6 s; CPU 6.0–6.4 s throughout. The GPU is within 1.2× of the CPU on a workload built for the CPU; it does not overtake it, and LIMITATIONS says so.
  • A10G (sm_86, CUDA 13.2): kernels.ptx regenerated (8 entries); parity 11/11 incl. the new index and fused-Adam tests; full workspace green with CUDA registered; compute-sanitizer memcheck 0 / racecheck 0 (whole suite, single-threaded; one parallel run showed 1 hazard that five reruns did not reproduce) / synccheck 0; train --device cuda --epochs 3 reaches loss 0.6693 (0.2.0's trajectory).

Lessons recorded

  • Waiting on the newest committed Metal command buffer is not enough; Metal finishes buffers out of commit order.
  • Re-registering a fresh backend from every test silently orphans encoded work once dispatch is asynchronous.
  • A downstream consumer (oxmega) found the fallback bug that the parity suite never exercised: the narrow VJP on a device tensor. It is a parity test now.

termlens moves alone (it pairs with nothing). 0.7.0 made Key and Signal
non_exhaustive (the CLI tests only construct them), 0.8.0 refuses resize
after the child exits; both doctor and dashboard suites, including the
100-iteration stresses, pass unchanged. MSRV 1.88 still checks against the
lockfile; cargo-deny green.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…, not upstream HEAD

reconverge is a rustc driver built on the nightly cuda-oxide needs; the
only cuda-oxide rev this project can take is the one the pinned reconverge
release records as verified. Upstream HEAD (97f8b2b7 at the time of #23,
fc3ec002 today) requires nightly-2026-08-28 while reconverge 0.4.0 is
built on nightly-2026-04-03, so the old watch reported drift nobody could
act on and would have re-filed it after every triage. The watch now reads
reconverge's conformance/PIN and rust-toolchain.toml at the pinned tag,
drifts only when those move, and prints HEAD for information. Run locally
against the current pins: no drift. Closes #23 together with the termlens
bump.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
… decay

ParamGroup { params, lr, weight_decay }; Sgd/Adam/AdamW/RmsProp gain
with_groups(...) and groups_mut() for schedules. The existing constructors
are the single-group case and produce identical updates (asserted). Adam's
bias-correction count stays global across groups. Closes #29.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
matmul now treats every leading dimension as a batch dimension with NumPy
broadcasting (right-aligned, 1 expands, a rank-2 operand is one matrix
for every batch). Ranks above 3 are lowered in the method layer onto the
unchanged rank-2/3 backend contract: broadcast (zero-stride view,
materialized only when a batch really repeats), flatten, multiply,
unflatten — every step a recorded op, so no new VJP. einsum(spec, &[a, b])
covers the common contractions ('ij,jk->ik', 'bij,bjk->bik',
'rbhd,rdo->rbho', 'ij->ji', 'i,i->', 'i,j->ij') through permute/sum/
matmul; diagonals and implicit output are typed errors. Loop-checked and
gradchecked. Closes #30.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…race, cholesky, logdet, det, eigh

eye/diag/diag_embed/trace/logdet/det are composites of recorded ops and
run and differentiate on every backend. cholesky and eigh are Backend
primitives with a CPU implementation (f64 internally, correctly rounded
f32 out) and the index_select-style CPU round-trip fallback for backends
that decline. cholesky carries Murray's (2016) VJP, computed on the host in
f64 and returned to the input's device; logdet's gradient therefore comes
out as the symmetrized inverse, checked against eigh. A non-PD matrix is a
typed error naming the batch index and pivot; eigh is cyclic Jacobi,
eigenvalues ascending, eigenvectors as columns, not differentiable.
Reconstruction, spectrum and gradient tests. Closes #26.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…l dispatch

index_select and index_add run on the device through two new kernels
(gather_dim, scatter_add_dim: one thread per output element scanning the
index list — deterministic, adds in index order like the CPU reference,
no atomics). The method-layer fallback now moves every operand through
the CPU round-trip: moving only the target left src on the device and
failed the narrow VJP with a DeviceMismatch (oxmega's k-DPP loss on
CUDA). kernels.ptx rebuilt from kernels.cu by nvcc 13.2 (compute_75, 7 entries).

Metal dispatch is asynchronous: ops are encoded into an open command
buffer (one encoder each, ordered with hazard tracking), committed every
64 ops or at the first host read, where every outstanding buffer is
awaited — Metal may finish command buffers out of commit order, so
waiting on the newest alone read zeros. register_default() is now
idempotent on Metal and CUDA: replacing a stateful backend mid-flight
left encoded work uncommitted (the parity suite under --test-threads).

Measured, oxmega device-bench (M4 Pro, 100 epochs × 3 stacked seeds,
back-to-back with the published 0.2.0, three rounds): Metal total
19.4–19.6 s → 7.5–7.7 s (2.6×); CPU 6.0–6.4 s both. Per fit: mlp-h16 BCE
1.50 → 0.33 s, linear-lag5 set-NLL 3.43 → 0.87 s, deepsets-H30 set-NLL
6.04 → 2.92 s.

A10G (sm_86, driver 595.71.05, CUDA 13.2, rustc 1.98): parity suite 10/10
incl. index ops with duplicate indices and strided views and narrow
backward on the device; full workspace tests green with CUDA registered;
compute-sanitizer memcheck 0 errors, racecheck 0 hazards, synccheck 0
errors over the index tests. Closes #25; first half of #28.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
One launch per parameter instead of the composite step's dozen tiny ops:
adam_step takes the parameter, gradient, optional moment state and the
step's scalars (AdamArgs, ten four-byte words shared with the kernels) and
returns (param, m, v). The optimizer uses it for parameters on a non-CPU
device and keeps the composite path as the reference and the fallback.
Parity tests hold the fused update to the composite CPU update over four
steps, Adam and AdamW, on both backends. ADR-0008 records the
asynchronous-dispatch decision, the out-of-order completion lesson and
the idempotent registration.

Measured (oxmega device-bench, M4 Pro, three rounds, back-to-back with
the published 0.2.0): Metal 19.4–20.1 s → 7.4–7.6 s with async dispatch
and the fused step (7.5–7.7 s with async alone); CPU 6.1–6.4 s.

A10G (sm_86, CUDA 13.2, rustc 1.98): kernels.ptx regenerated (8 entries);
parity 11/11; full workspace green with CUDA registered; compute-sanitizer
memcheck 0 errors, racecheck 0 hazards on the whole suite single-threaded
and on the adam/index tests in five reruns (one parallel run reported 1
hazard that no rerun reproduced), synccheck 0 errors; oxmera train
--device cuda --epochs 3 reaches loss 0.6693 (identical to 0.2.0's
trajectory) at ~47k samples/s. Closes #28.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…utograd

DType::F64 storage (CpuStorage::F64), from_vec_f64/to_vec_f64/get_f64,
and Tensor::to_dtype (F32 ↔ F64, I64 → float; differentiable, the gradient
converts back). The CPU backend runs every op family in f64 — unary,
broadcasting binary, matmul, reductions (Neumaier-compensated sums),
argmax, index_select/index_add, contiguous, cholesky, eigh — through a
plain reference path (cpu_f64.rs) that keeps the tuned f32 kernels
untouched. VJP plumbing follows the dtype (scalar_on, the narrow and
index_select zeros, the backward seed, eye_like), so a loss evaluated in
f64 differentiates in f64 and an f32 leaf evaluated through to_dtype gets
an f32 gradient. The GPU backends stay f32: to_device of an f64 tensor and
mixed-dtype arithmetic are typed errors. Closes #27.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…e milestone

Every number in the changelog and LIMITATIONS comes from this session's
measurements (oxmega device-bench before/after, the A10G parity and
sanitizer logs). The Metal performance paragraph now says what was
measured: within 1.2x of the CPU on the motivating benchmark, not faster.

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…nsum, f64, param groups (#23, #25#30)

Signed-off-by: Vyncint Ng <vyncint@icloud.com>
Signed-off-by: Vyncint Ng <vyncint@icloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pins: upstream drift — termlens@0.8.0 cuda-oxide@97f8b2b7

1 participant