Per-parameter-group weight decay / learning rate in the optimizers

[vyncint]

Where it bites

oxmega trains seeds as stacked replicas in one graph (one parameter slab per seed) because oxmera's batch-broadcast matmul makes that free. The natural next step — stacking the hyper-parameter grid the same way (weight decay ∈ {0.01, 0.1, 1}) — is blocked because AdamW::new(params, lr, wd) takes one scalar for all parameters.

Proposal

AdamW::with_groups(&[(params, lr, wd)]) or a per-Param override (Param::with_weight_decay). Elementwise Adam is already independent per element, so a per-parameter (or per-leading-index) decay tensor is the minimal change: wd broadcastable against the parameter shape.

[vyncint]

Shipped in oxmera 0.3.0 (merge 0683ca0, PR #31; all twelve crates on crates.io): ParamGroup { params, lr, weight_decay }, with_groups(..) on Sgd/Adam/AdamW/RmsProp, groups_mut() for schedules; the single-group constructors produce identical updates (asserted). Details and measurements in the 0.3.0 changelog.