Conversation
termlens moves alone (it pairs with nothing). 0.7.0 made Key and Signal non_exhaustive (the CLI tests only construct them), 0.8.0 refuses resize after the child exits; both doctor and dashboard suites, including the 100-iteration stresses, pass unchanged. MSRV 1.88 still checks against the lockfile; cargo-deny green. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…, not upstream HEAD reconverge is a rustc driver built on the nightly cuda-oxide needs; the only cuda-oxide rev this project can take is the one the pinned reconverge release records as verified. Upstream HEAD (97f8b2b7 at the time of #23, fc3ec002 today) requires nightly-2026-08-28 while reconverge 0.4.0 is built on nightly-2026-04-03, so the old watch reported drift nobody could act on and would have re-filed it after every triage. The watch now reads reconverge's conformance/PIN and rust-toolchain.toml at the pinned tag, drifts only when those move, and prints HEAD for information. Run locally against the current pins: no drift. Closes #23 together with the termlens bump. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
… decay
ParamGroup { params, lr, weight_decay }; Sgd/Adam/AdamW/RmsProp gain
with_groups(...) and groups_mut() for schedules. The existing constructors
are the single-group case and produce identical updates (asserted). Adam's
bias-correction count stays global across groups. Closes #29.
Signed-off-by: Vyncint Ng <vyncint@icloud.com>
matmul now treats every leading dimension as a batch dimension with NumPy
broadcasting (right-aligned, 1 expands, a rank-2 operand is one matrix
for every batch). Ranks above 3 are lowered in the method layer onto the
unchanged rank-2/3 backend contract: broadcast (zero-stride view,
materialized only when a batch really repeats), flatten, multiply,
unflatten — every step a recorded op, so no new VJP. einsum(spec, &[a, b])
covers the common contractions ('ij,jk->ik', 'bij,bjk->bik',
'rbhd,rdo->rbho', 'ij->ji', 'i,i->', 'i,j->ij') through permute/sum/
matmul; diagonals and implicit output are typed errors. Loop-checked and
gradchecked. Closes #30.
Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…race, cholesky, logdet, det, eigh eye/diag/diag_embed/trace/logdet/det are composites of recorded ops and run and differentiate on every backend. cholesky and eigh are Backend primitives with a CPU implementation (f64 internally, correctly rounded f32 out) and the index_select-style CPU round-trip fallback for backends that decline. cholesky carries Murray's (2016) VJP, computed on the host in f64 and returned to the input's device; logdet's gradient therefore comes out as the symmetrized inverse, checked against eigh. A non-PD matrix is a typed error naming the batch index and pivot; eigh is cyclic Jacobi, eigenvalues ascending, eigenvectors as columns, not differentiable. Reconstruction, spectrum and gradient tests. Closes #26. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…l dispatch index_select and index_add run on the device through two new kernels (gather_dim, scatter_add_dim: one thread per output element scanning the index list — deterministic, adds in index order like the CPU reference, no atomics). The method-layer fallback now moves every operand through the CPU round-trip: moving only the target left src on the device and failed the narrow VJP with a DeviceMismatch (oxmega's k-DPP loss on CUDA). kernels.ptx rebuilt from kernels.cu by nvcc 13.2 (compute_75, 7 entries). Metal dispatch is asynchronous: ops are encoded into an open command buffer (one encoder each, ordered with hazard tracking), committed every 64 ops or at the first host read, where every outstanding buffer is awaited — Metal may finish command buffers out of commit order, so waiting on the newest alone read zeros. register_default() is now idempotent on Metal and CUDA: replacing a stateful backend mid-flight left encoded work uncommitted (the parity suite under --test-threads). Measured, oxmega device-bench (M4 Pro, 100 epochs × 3 stacked seeds, back-to-back with the published 0.2.0, three rounds): Metal total 19.4–19.6 s → 7.5–7.7 s (2.6×); CPU 6.0–6.4 s both. Per fit: mlp-h16 BCE 1.50 → 0.33 s, linear-lag5 set-NLL 3.43 → 0.87 s, deepsets-H30 set-NLL 6.04 → 2.92 s. A10G (sm_86, driver 595.71.05, CUDA 13.2, rustc 1.98): parity suite 10/10 incl. index ops with duplicate indices and strided views and narrow backward on the device; full workspace tests green with CUDA registered; compute-sanitizer memcheck 0 errors, racecheck 0 hazards, synccheck 0 errors over the index tests. Closes #25; first half of #28. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
One launch per parameter instead of the composite step's dozen tiny ops: adam_step takes the parameter, gradient, optional moment state and the step's scalars (AdamArgs, ten four-byte words shared with the kernels) and returns (param, m, v). The optimizer uses it for parameters on a non-CPU device and keeps the composite path as the reference and the fallback. Parity tests hold the fused update to the composite CPU update over four steps, Adam and AdamW, on both backends. ADR-0008 records the asynchronous-dispatch decision, the out-of-order completion lesson and the idempotent registration. Measured (oxmega device-bench, M4 Pro, three rounds, back-to-back with the published 0.2.0): Metal 19.4–20.1 s → 7.4–7.6 s with async dispatch and the fused step (7.5–7.7 s with async alone); CPU 6.1–6.4 s. A10G (sm_86, CUDA 13.2, rustc 1.98): kernels.ptx regenerated (8 entries); parity 11/11; full workspace green with CUDA registered; compute-sanitizer memcheck 0 errors, racecheck 0 hazards on the whole suite single-threaded and on the adam/index tests in five reruns (one parallel run reported 1 hazard that no rerun reproduced), synccheck 0 errors; oxmera train --device cuda --epochs 3 reaches loss 0.6693 (identical to 0.2.0's trajectory) at ~47k samples/s. Closes #28. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…utograd DType::F64 storage (CpuStorage::F64), from_vec_f64/to_vec_f64/get_f64, and Tensor::to_dtype (F32 ↔ F64, I64 → float; differentiable, the gradient converts back). The CPU backend runs every op family in f64 — unary, broadcasting binary, matmul, reductions (Neumaier-compensated sums), argmax, index_select/index_add, contiguous, cholesky, eigh — through a plain reference path (cpu_f64.rs) that keeps the tuned f32 kernels untouched. VJP plumbing follows the dtype (scalar_on, the narrow and index_select zeros, the backward seed, eye_like), so a loss evaluated in f64 differentiates in f64 and an f32 leaf evaluated through to_dtype gets an f32 gradient. The GPU backends stay f32: to_device of an f64 tensor and mixed-dtype arithmetic are typed errors. Closes #27. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
…e milestone Every number in the changelog and LIMITATIONS comes from this session's measurements (oxmega device-bench before/after, the A10G parity and sanitizer logs). The Metal performance paragraph now says what was measured: within 1.2x of the CPU on the motivating benchmark, not faster. Signed-off-by: Vyncint Ng <vyncint@icloud.com>
Signed-off-by: Vyncint Ng <vyncint@icloud.com>
This was referenced Sep 3, 2026
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
0.3.0 — the "what oxmega needs" milestone, plus the pins triage
Closes #23, #25, #26, #27, #28, #29, #30.
What changed
a766fc26: upstream HEAD requiresnightly-2026-08-28, reconverge 0.4.0 is a rustc driver built onnightly-2026-04-03and itsconformance/PINstill verifiesa766fc26. The pins watch now compares against what the pinned reconverge tag records instead of upstream HEAD, so a drift issue means something this project can act on. Run locally against the current pins: no drift.index_select/index_addrun on Metal and CUDA (gather_dim,scatter_add_dim: one thread per output element scanning the index list — deterministic, adds in index order like the CPU, no atomics). The fallback now moves every operand through the CPU round-trip; moving only the target failed thenarrowVJP with a DeviceMismatch on CUDA (oxmega's k-DPP loss).eye/eye_on,diag,diag_embed,trace,cholesky(Murray-2016 VJP, host f64),logdet,det,eigh(Jacobi).cholesky/eighare backend primitives with a CPU implementation and the index-op fallback contract; non-PD is a typed error naming the batch.DType::F64on the CPU end to end (storage, every primitive, autograd,to_dtype); GPUs stay f32 with typed errors.adam_stepkernel on both GPU backends.register_default()is idempotent: a stateful backend must not be replaced mid-flight. ADR-0008.ParamGroup { params, lr, weight_decay },with_groupson every optimizer,groups_mut()for schedules; single-group constructors produce identical updates (asserted).einsumfor one- and two-operand contractions.Measured
kernels.ptxregenerated (8 entries); parity 11/11 incl. the new index and fused-Adam tests; full workspace green with CUDA registered; compute-sanitizer memcheck 0 / racecheck 0 (whole suite, single-threaded; one parallel run showed 1 hazard that five reruns did not reproduce) / synccheck 0;train --device cuda --epochs 3reaches loss 0.6693 (0.2.0's trajectory).Lessons recorded
narrowVJP on a device tensor. It is a parity test now.