minimax_music3: performance pack — native rope/SwiGLU lowering + opt-in CFG reuse, chunk hop, batched ensemble takes, q4_k depth

[iamwavecut]

What

A performance pack for MiniMax Music 3 (improving the existing family per CONTRIBUTING, building on #241/#243/#271 by @JoeMattie). 13 focused commits; out of the box only the flow lowering changes — everything else is opt-in and the defaults reproduce the current upstream trajectory.

Default-on (changes flow internals, spectrally equivalent):

• Native ggml_rope_ext NEOX lowering instead of the slice/concat rope chains, both flow batches. Also works around a gallocr view-aliasing hazard we bisected: a freshly allocated batch-1 flow graph corrupts its first ~4 invocations, bitwise-reproducibly (batch 2 is saved by an explicit ggml_repeat). Repro/diagnostics ship as MM3_DC_B1_* envs; MM3_LEGACY_ROPE=1 restores the old lowering.
• Fused ggml_swiglu_swapped for the flow FFN (our layout is [states|gate] — plain ggml_swiglu silences the wrong half, mel 25 dB off; the swapped variant is exact). MM3_LEGACY_GLU=1 restores.
• Compact semantic LM head (exact quantized-row repack, byte-identical sampling — gate below), CUDA graph-cache eviction on release, pipeline buffer reuse.

Opt-in (upstream defaults preserved):

• flow_uncond_interval=N — reuse the CFG guidance delta between steps. N=2: flow −25-30%, mel-L1 ~0.3 dB vs exact; N=1 (default) = exact trajectory.
• flow_chunk_hop_frames — the stock chunking (200-frame chunks, hop 100) denoises every second twice. hop=150: flow −35%; crops and the carry window are rederived consistently (hop=100 reproduces the stock geometry byte-for-byte).
• ensemble_takes=K (1..16) — decode K takes of one prompt in a single batched AR pass. The LM+depth stages are weight-bandwidth-bound, so K takes cost far less than K runs; flow/vocoder run per take, outputs land as take_NN via --out-dir. Adds an additive export_batched_decode_state() to the Qwen decode runtime (no behavior change for existing callers).
• ensemble_prefix_frames=N — intro-lock: the first N frames decode once as a shared master trajectory, then takes fork (take_01 continues the master exactly). "One intro, K continuations."
• Depth decoder q4_k (upstream file): −24% depth-stage time, panel-clean; option only, default stays q8_0.

Numbers (fresh run on a clean A40 SECURE pod, driver 580.159.04, this branch, CUDA)

60 s track, 30 flow steps, one seed, single run each (fresh clone of this branch, canonical build script):

run	wall	vs defaults
def60 — upstream defaults	95.9 s	baseline
slim60 — compact LM head only	88.8 s	−7.4% (output byte-identical, see gate)
dc2_60 — flow_uncond_interval=2	87.1 s	−9.2%
hop150_60 — flow_chunk_hop_frames=150	79.4 s	−17.2%
combo60 — interval=3 + hop=150 + q4_k depth + slim head	68.3 s	−28.8%
ens6 — ensemble_takes=6, combo options	223.4 s = 37.2 s/song	−61% per song

20 s sanity pair: native lowering 33.9 s vs MM3_LEGACY_ROPE/GLU 36.2 s (−6% from the lowering itself).

From the longer campaign behind this: the ensemble K-curve on a 60 s track runs 89.3/72.4/67.2/64.4/63.7 s per AR pass for K=1/3/6/10/16 (LM+depth are weight-bandwidth-bound, so extra takes are nearly free until flow dominates). Every recipe step was gated on a spectral panel (chroma/harmonicity/dynamics/onsets staying in the reference band) plus human listening.

Build / run

scripts/build_linux.sh --build-dir build/rel --build-type Release --backend cuda --cuda-arch native \
  --model-set custom --models minimax_music3 --with-tests --target audiocpp_cli --jobs $(nproc)

Baseline run (upstream defaults, only the flow lowering differs):

build/rel/audiocpp_cli --task gen --family minimax_music3 --model <models> --backend cuda \
  --text "<caption>" --request-option "lyrics=<lyrics>" --request-option duration_sec=60 \
  --request-option num_inference_steps=30 --request-option seed=20260827 --metrics --out out.wav

Full opt-in recipe (what combo60 in the table runs):

... same as above, plus:
  --request-option flow_uncond_interval=3 --request-option flow_chunk_hop_frames=150 \
  --session-option minimax_music3.rvq_depth_decoder_gguf=rvq_depth_decoder_q4_k.gguf \
  --session-option minimax_music3.language_model_gguf=language_model_q4_0_slim.gguf

Ensemble: add --request-option ensemble_takes=6 --out-dir out/ (takes land as take_NN.wav).

Validation

• minimax_music3_graph_release_policy_test, minimax_music3_lm_head_test, minimax_music3_pipeline_buffers_test: pass.
• Slim head: output byte-identical to the full head (sha256 gate in the fresh run).
• hop=100 path: byte-identical geometry to stock.

Spectrograms from the fresh run (branch evidence/mm3-perf-20260827 holds these + the raw table):

defaults (95.9 s) vs full opt-in recipe (68.3 s), same seed:

legacy lowering vs native lowering (20 s, same seed — diverged takes, same character):

Known limitations

• Native rope/SwiGLU are not byte-identical to the previous lowering. On matched token streams the flow output sits at mel-L1 ~0.15 dB; on a full same-seed run the AR sampler eventually picks different tokens (any numeric shift in the loop does this), so tracks diverge into equally-valid takes (~0.8 dB log-spec apart, same spectral character — spectrograms above). MM3_LEGACY_ROPE/GLU keep the exact old path if you need it.
• Validated on CUDA only; we saw minimax_music3: degenerate (unconditioned, drone-like) output on Metal — runs at full speed, no errors #303 (Metal) and did not touch Metal-specific paths, but flow lowering changes should be re-checked there.
• ensemble_takes peak VRAM grows ~0.7 GB per take at 60 s (K=6 ≈ 12.4 GB total on A40).
• The MMQ fixup-buffer memset also ships as a standalone micro-PR; whichever lands first, the other merges clean (identical hunk).

[0xShug0]

@iamwavecut Thanks for the PR. One issue I recently found is that some optimizations were rejected by me because, on my setup, they only showed noise-level performance changes, while on different hardware the gains could be significant. So I may miss many useful optimizations.

RTX 5090:

Case	Wall ms	RTF	Idle VRAM	Peak VRAM
main default	12802.5	0.7652	2232 MB	12576 MB
PR default	12522.9	0.7501	2224 MB	12564 MB
main mem_saver=false	10693.3	0.6392	15146 MB	16714 MB
PR mem_saver=false	10289.0	0.6163	15142 MB	16712 MB
main mem_saver=true	12874.5	0.7695	2232 MB	12576 MB
PR mem_saver=true	12499.0	0.7487	2224 MB	12564 MB

I’m not a CI expert, so I’m not sure whether the new cuBLASLt dependency could break Docker images or prebuilts. I need a bit more time to investigate.

[0xShug0]

Also I think the changes to external/ggml/src/ggml-cuda/mmq.cuh was merged already.

[iamwavecut]

Fair points. Dropped the cuBLASLt commit entirely — it never beat MMQ in my runs either, not worth a new dependency — and rebased on main, so the mmq.cuh hunk is gone too (you already merged it). 12 commits now.

On the 2–4%: your run only exercises the default-on part. The bigger wins are the opt-in knobs (flow_uncond_interval, flow_chunk_hop_frames, ensemble_takes), and they're largest on bandwidth-bound cards like A40/3090 — a 5090 hides most of that.

[0xShug0]

@iamwavecut Thanks for the update! The opt-in options help a lot. I just picked two that don't require new GGUFs. Songs are valid based on my ears.

Case	PR opt-in options	Wall ms	Output sec	RTF	Idle VRAM	Peak VRAM
main default	none	55832.3	60.07	0.929453	431 MB	11092 MB
PR default	flow interval 3, hop 150	35025.7	55.93	0.626294	409 MB	11069 MB
main mem_saver=false	none	45584.8	60.07	0.758861	431 MB	12730 MB
PR mem_saver=false	flow interval 3, hop 150	33526.5	55.93	0.599487	409 MB	12707 MB
main mem_saver=true	none	46930.8	60.07	0.781269	431 MB	11092 MB
PR mem_saver=true	flow interval 3, hop 150	34950.9	55.93	0.624957	409 MB	11069 MB

Please address these issues:

1. Looks like the useful runtime env variables are MM3_LEGACY_ROPE, MM3_LEGACY_GLU, MM3_DEPTH_GPU_SAMPLE, MM3_DEPTH_GPU_FRAME, and GGML_CUDA_STREAM_PRIORITY. The others appear to be debug/probe controls.
   I’d prefer to keep the final code clean by removing state-dump and debug-only paths, so the code is easier to read and used as "template" for future similar models.

2. GGML_CUDA_STREAM_PRIORITY is a generic, process-wide ggml env var. Even though this PR sets and unsets it only around MiniMax Music3 overlap setup, another ggml CUDA stream created during that window could accidentally inherit the priority. A scoped additive backend/API mechanism would be cleaner.

3. The Music3 docs (https://github.com/0xShug0/audio.cpp/blob/main/docs/community_models/minimax_music3.md) should be updated for the new MiniMax Music3 request/session options, opt-in GGUFs and any env variables intended to remain supported.

[iamwavecut]

Done, all three:

1. Debug paths — stripped entirely (the MM3_DC_* probes, the carry-free flow probe, the K=1 ensemble force). Net −220 lines. Supported env vars are now exactly the four you listed: MM3_LEGACY_ROPE, MM3_LEGACY_GLU, MM3_DEPTH_GPU_SAMPLE, MM3_DEPTH_GPU_FRAME.

2. Stream priority — the GGML_CUDA_STREAM_PRIORITY env is gone. It's now a per-backend field with ggml_backend_cuda_set_stream_priority(), resolved through the proc-address table like clear_graph/trim_pools. Nothing outside the target backend instance can inherit the priority anymore.

3. Docs — minimax_music3.md now covers the opt-in options with defaults and measured wins, pipeline_overlap, the q4_k depth GGUF, and the supported env vars.

Smoke-tested on the official GGUF pack: default, interval+hop, ensemble, overlap, legacy env, q4_k depth — all pass.

[0xShug0]

@iamwavecut Merged. Thanks! 🎉