Skip to content

feat(engine): spectral control backend (on models/backend)#232

Merged
ryanontheinside merged 3 commits into
mainfrom
ryanontheinside/feat/models/spectral/spectral-control
Jun 9, 2026
Merged

feat(engine): spectral control backend (on models/backend)#232
ryanontheinside merged 3 commits into
mainfrom
ryanontheinside/feat/models/spectral/spectral-control

Conversation

@ryanontheinside

Copy link
Copy Markdown
Collaborator

Spectral control backend (activation steering), ported onto the backend seam

Merge order

Stacked: opened against ryanontheinside/feat/models/backend. Merge #224 -> #220 -> #229 before this.
Stack: main <- #224 <- #220 <- #229 (models/backend) <- this <- spectral demo PR. Sibling leaves off models/backend: mrt2 (#230), sa3 (#231) — this PR does not touch families.py, so it composes cleanly with both; the demo PR stacked on top carries the families.py heads-up.
Supersedes #162 (same feature, ported from the pre-seam ryanontheinside/feat/spectral/spectral-control; close that one in favor of this).

Engine-side activation steering: per-DiT-layer additive shifts on the post-block residual, gated per-row by denoise step, driven live from the streaming pipeline.

What's here (one commit, 39b83c3)

  • acestep/steering/: new package — SteeringController (auto-axis vectors + manual catalog + LIFO slot registry + raw-knobs -> config translation), policy.py (pure axis/limits tables), catalog.py (vector enumeration/loading), hub.py (per-checkpoint HF bundle fetch/cache), types.py.
  • acestep/engine/stream.py: set_steering() API; eager-path forward hooks on decoder.layers; TRT-path per-tick steering buffer fill; _current_step_per_row populated around the batched forwards so injection only fires at the targeted denoise step.
  • acestep/engine/trt/export.py + build.py + runtime.py: decoder ONNX/engines gain a [B, num_layers, hidden_size] steering input (zeros = exact no-op). Engine filename prefix becomes spectral_decoder_*; pre-steering engines keep their old names and the streaming path degrades with a loud warning (knobs no-op until rebuilt).
  • acestep/paths.py: steering-vector bundle paths per checkpoint; TRT profile table points at the spectral_decoder_* engines.
  • Engine metadata schema bumped 1 -> 2 (decoder contract changed shape).

Port notes (vs #162)

  • stream.py lands on the ModelAdapter seam: the per-row step mapping wraps _forward_pairs, which now dispatches through adapter.batched_forward (ACE) — TRT dispatch and the steering buffer fill stay pipeline-owned, so the injection semantics are unchanged.
  • _ENGINE_METADATA_SCHEMA moved to its post-refactor home acestep/engine/trt/_engine_metadata.py; the 1 -> 2 bump lives there now.
  • paths.py keeps both the seam's fixtures/user-uploads dirs and the steering helpers.

Validation

  • 149/149 unit tests at this tip (full inherited guard suite).
  • Golden harness: pending a free GPU window; will be run on the demo tip (which contains this commit) and reported there.

Base automatically changed from ryanontheinside/feat/models/backend to main June 8, 2026 13:59
Also relabels the two sections inside as "channel highlights" and
"channel groups" so the section ↔ tab vocabulary stays honest.
^ Conflicts:
^	demos/realtime_motion_graph_web/web/sdk/protocol.ts
@ryanontheinside ryanontheinside force-pushed the ryanontheinside/feat/models/spectral/spectral-control branch from 6dbb2fc to 07f8759 Compare June 9, 2026 11:46
@ryanontheinside ryanontheinside merged commit 7659a7c into main Jun 9, 2026
@ryanontheinside ryanontheinside deleted the ryanontheinside/feat/models/spectral/spectral-control branch June 9, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant