Skip to content

refactor(optimizer)!: unify SGD step arms + updateMath knob (#308, #310)#313

Merged
LeoBuron merged 2 commits into
developfrom
sgd-unify-updatemath
Jul 8, 2026
Merged

refactor(optimizer)!: unify SGD step arms + updateMath knob (#308, #310)#313
LeoBuron merged 2 commits into
developfrom
sgd-unify-updatemath

Conversation

@LeoBuron

@LeoBuron LeoBuron commented Jul 8, 2026

Copy link
Copy Markdown
Member

Two-issue stack on the SGD update path, executed via SDD (per-task reviews + final whole-branch review, verdict: merge-ready).

#308 — unify SGD step arms

  • sgdStepM gains a momentum==0 fast path: one stateless executeOp per parameter (param -= lr*(grad + wd*param)), mathematically identical to the two-op momentum path at momentum==0, and never touches optim->states.
  • sgdMCreateOptim allocates no momentum-state buffers at momentumFactor==0 (optim->states == NULL) — real RAM back on MCU targets; freeOptimSgdM handles the no-states case. The ten duplicated per-layer state-building blocks collapsed into one uniform post-loop (−73 lines).
  • The unreachable plain-SGD arm is gone: sgdStep deleted, optimizerType_t is now { SGD_M } (verified unserialized anywhere), vtable has one entry. Its three direct test callers (the mutation-hardened optimizer: SYM_INT32 param/momentum update has a fixed-scale dead-zone (sub-ULP updates vanish) #279 dead-zone fixtures) are ported to the unified step with fixture math, iteration counts, seeds, and assertions byte-identical.
  • New semantic note: momentumFactor is creation-fixed — it now decides state allocation (documented in SgdApi.h).

#310updateMath knob

  • sgdMCreateOptim gains a 7th parameter, by-value arithmetic_t updateMath, stored on sgd_t and fed into all three update opSpecs (previously hardcoded {ARITH_FLOAT32, HALF_AWAY} — a SYM-arithmetic update was structurally impossible).
  • Default {ARITH_FLOAT32, HALF_AWAY} = zero numeric movement; all ~48 test call sites + 9 example call sites swept mechanically, no expectation changed anywhere.
  • Anything but ARITH_FLOAT32 fails fast at creation (sgdInit) AND at step time (sgdStepM entry, protects hand-assembled optimizers — the update kernels raw-cast float*). Both guards death-tested; the creation death test doubles as an arg-forwarding pin.
  • Rounding ownership documented at the arithmetic/executeOp: rounding mode is storage-owned (quantization) but is conceptually operation-owned #282 seam: updateMath.roundingMode governs the funnel prologue (inert for FLOAT32); the OUT_WRITE epilogue rounds by each target's own qConfig — where the optimizer: SYM_INT32 param/momentum update has a fixed-scale dead-zone (sub-ULP updates vanish) #279 SR dead-zone escape lives.
  • Known accepted limit (from the plan): with only FLOAT32 admitted, the opSpec feed lines have no behaviorally distinguishing test; they are pinned by the two guards + a 7-arg _Static_assert signature contract. The first real second arm owns the distinguishing tests.

Also in this PR

Verification

Closes #308
Closes #310

Note: develop-merge does not auto-close — close both manually after merge. Follow-up to file: sizeStates rename (doubles as param count; misnomer sharpened now that states can be NULL).

🤖 Generated with Claude Code

LeoBuron and others added 2 commits July 8, 2026 19:31
…ateless factory, plain-SGD arm deleted (#308)

- sgdStepM: momentumFactor==0 runs one stateless update op per parameter and
  never reads optim->states (may be NULL)
- sgdMCreateOptim: allocates no momentum-state buffers at momentumFactor==0
  (optim->states == NULL); freeOptimSgdM handles the no-states case; the ten
  duplicated per-layer state-building blocks DRYed into one uniform post-loop
- delete unreachable sgdStep + SGD enum value (verified unserialized); its
  three direct test callers (#279 dead-zone fixtures) ported byte-identically

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…GD update ops, FLOAT32 default, fail-fast otherwise (#310)

- sgdMCreateOptim gains a by-value arithmetic_t updateMath (7th param, mirrors
  the layer-side per-op knobs), stored on sgd_t and fed into all three update
  opSpecs (previously hardcoded {ARITH_FLOAT32, HALF_AWAY})
- anything but ARITH_FLOAT32 fails fast at creation (sgdInit) AND step time
  (sgdStepM entry; the update kernels raw-cast float*) — both death-tested
- mechanical sweep: 48 test + 9 example call sites, zero numeric movement;
  7-arg _Static_assert signature contract updated
- rounding ownership documented at the #282 seam (prologue = knob, inert for
  FLOAT32; OUT_WRITE epilogue = target qConfig, the #279 SR escape)
- FEATURES.md optimizer section corrected; stale pre-#284 comments swept;
  mem_instrument guards states==NULL; momentumFactor creation-fixed doc note

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant