[RFC][MiniMax-H3][CUDA/ROCm] WS1/WS2 kernel roadmap, ablation matrix and integration plan

[maxiaosong1124]

Status: Proposed

Target: post-v0.1.0 community roadmap

Related: #386, #415

Reference Qwen3-8B Dense track: #180, #204, #228, #230, #240, #243, #280, #315, #322, #336, #338, #343, #352, #360, #361, #377.

Upstream checkpoint pinned for this draft: MiniMaxAI/MiniMax-H3@42ed227ee7df40d41602854ae760620d6eb651fe.

---

1. Motivation

RL-Kernel has closed the Qwen3-8B Dense consistency loop from deterministic WS1 operators through WS2 TP/CP/SP execution, runtime provenance, ablation, VIME/vLLM integration, and an end-to-end distributed experiment.

MiniMax-H3 is a different consistency target. It is a 33B dense, single-stream audio-video flow model rather than an autoregressive language model. The released model packs text, video, and audio rows into one full-attention document, applies a 50-layer Omni-Transformer, predicts video and audio velocities, and advances two deterministic rectified-flow schedules.

The strict H3 objective is therefore:

Given the same checkpoint, encoded conditions, latent inputs, packed-row identity, modality tags, timestep table, three-axis positions, sigma schedules, and initial noise, training and rollout must execute the same declared arithmetic contract and produce exactly equal video/audio velocities and denoising trajectories in strict mode.

Selected-token log-probability is not the primary acceptance surface for the released H3 sampler. The public scheduler uses eta = 0, so it defines a deterministic ODE update and does not by itself define a non-degenerate Gaussian transition log-probability. A future stochastic RL sampler may add a transition-logprob contract, but that work must declare the SDE/noise law first and must not label an ODE trajectory score as a log-probability.

The roadmap is split into independently closable phases:

• Phase A - Omni-Transformer core: pre-encoded text, video latents, and audio latents; packed-sequence construction; WS1 operators; WS2 distribution; deterministic dual-scheduler trajectory.
• Phase B - released end-to-end pipelines: Qwen3-VL text encoder, VisualVAE, AudioVAE, FL2VA and Ref2VA integration, and media decode validation.
• Phase C - RL integration: training/rollout adapters, LoRA/full-parameter gradient validation, trajectory-level reward experiments, and an optional stochastic transition-logprob path after its probability model is specified.

This ordering keeps the arithmetic core unblocked by the much larger encoder/VAE and task-orchestration surface.

Scope boundary

The open-source release does not include H3-Context-IR, H3-Regenerate-2K, or the announced native sparse-attention implementation. Those components are out of scope until reproducible public code and weights exist. The initial strict attention target is the released full-attention path.

---

2. Checkpoint fingerprint

Every validation artifact must record the exact checkpoint revision, runtime revision, task family, scheduler settings, dtype policy, and backend identities. This draft is based on MiniMaxAI/MiniMax-H3@42ed227ee7df40d41602854ae760620d6eb651fe.

Omni-Transformer fingerprint

Item	Value
Architecture	MiniMaxH3Transformer3DModel
Model family	33B dense single-stream audio-video flow transformer
Transformer dtype	BF16, with declared FP32 modules listed below
Residual hidden size	5376
Attention heads / head dim	56 / 128
Attention inner dimension	7168
Transformer layers	50
Text-refiner layers	2
FFN hidden size	14336
FFN activation	SwiGLU
Attention	full bidirectional self-attention, no cross-attention
Q/K preprocessing	per-head RMSNorm, epsilon 1e-5
Position encoding	three-axis (t, h, w) MM-RoPE
RoPE frequencies	16 per axis, theta 10000
Rotary channels	96 of 128 head channels; remaining 32 pass through
Block norm	RMSNorm, epsilon 1e-5
AdaLN modalities	3: video=0, text=1, audio=2
AdaLN conditioning	row = timestep_index * 3 + token_tag
AdaLN projection	2688 -> 6 * 3 * 5376 = 96768 per block
Dropout	0

Input, output and conditioning fingerprint

Item	Value
Text encoder	full Qwen3-VL-32B weights; layer-50 hidden state
Text feature width	5120
Text projection	5120 -> 5376, then 2-layer token refiner
Visual VAE latent	24 channels, spatial factor 16, temporal factor 4
Transformer video patch	(1, 2, 2), flattened input width 96
Effective visual token compression	spatial factor 32, temporal factor 4
Video input/output heads	96 -> 5376 and 5376 -> 96
Audio VAE latent	32 channels at 40 Hz per stereo channel path
Audio input/output heads	32 -> 5376 and 5376 -> 32
Timestep sinusoid width	256
Timestep MLP	256 -> 5376 -> 2688
Packed layout	text, video, and audio rows scattered by explicit index tensors
Batch semantics	pure replication axis; one structural layout shared by all batch rows
Output selection	video_indices and audio_indices after both heads run over the packed sequence

Mixed-precision fingerprint

The checkpoint is intentionally mixed precision. Strict mode must preserve these boundaries rather than normalizing the whole graph to BF16:

FP32 module/path	BF16 module/path
video input projection	text projection and token refiner
audio input projection	50-block transformer stack
timestep MLP	per-block AdaLN projections
video output projection	Q/K/V/O and FFN projections
audio output projection	RMSNorm and Q/K norm outputs after declared casts
RoPE frequency construction	packed residual stream

Scheduler fingerprint

Item	Value
Sampler family	deterministic rectified-flow Euler, eta = 0
Video shift	12.0 by default
Audio shift	3.0 by default
Sigma transform	s * sigma / (1 + (s - 1) * sigma)
Timestep convention	t = 1 - sigma, with t = 1 clean
Velocity convention	data-ward, x0 = xt + sigma * v
Euler update	x_next = r * xt + (1 - r) * x0, r = sigma_next / sigma
Grid	shifted linspace(1, 0, steps), consecutive FP32 duplicates removed
Model evaluations	len(sigmas) - 1

Released variants

Variant	Transformer	Input mode
FL2VA	transformer/	text with zero, first, last, or first-and-last frame conditioning
Ref2VA	transformer_ref/	text plus reference image, video, and/or audio inputs

Upstream references

• https://huggingface.co/MiniMaxAI/MiniMax-H3
• https://github.com/MiniMax-AI/MiniMax-H3
• https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_minimax_h3.py
• https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_minimax_h3.py

Upstream runtime status as of 2026-09-17

• Diffusers provides the executable model, scheduler, modular pipeline, context-parallel plan, and released component mappings. It is the initial reference runtime.
• The MiniMax model card recommends SGLang, vLLM, Diffusers, and ComfyUI. Exact H3 versions, supported task families, and injection/readback hooks must be fingerprinted again when each adapter PR starts.
• RL-Kernel currently has no MiniMax-H3 semantic operator plan, checkpoint mapper, trajectory comparator, or training provider. These are new integration items rather than assumed reuse.

---

3. Architecture map

Figure 1 - MiniMax-H3 released path and consistency boundaries. Phase A begins after the text encoder and VAEs, closes the packed Omni-Transformer and both deterministic schedules, and produces exact latent trajectories. Phase B connects the released encoders, VAEs, task adapters, and decoders.

The central difference from Qwen3-8B is that one model call processes three modalities and potentially several distinct timesteps in one sequence. Logical row identity is the tuple:

(batch item, packed row, modality, timestep table row, t/h/w coordinate, source index, output-selection index).

Every member of that tuple must remain aligned under packing, batching, TP, CP, SP, checkpointing, and runtime adaptation.

One Omni-Transformer block

Figure 2 - One H3 block. WS1 qualifies the arithmetic at each boundary. WS2 preserves the same graph while sharding attention heads, FFN channels, AdaLN output channels, and packed sequence rows with deterministic communication.

Two details require dedicated validation rather than generic transformer coverage:

1. 56 * 128 = 7168, so attention projections expand beyond the 5376-wide residual stream and the TP plan cannot assume num_heads * head_dim == hidden_size.
2. MM-RoPE rotates only 96 head channels. Applying standard full-head RoPE is a plausible implementation error that may remain numerically stable while producing a different trajectory.

---

4. Numerical contract

The existing RL-Kernel WS1 numerical standard remains authoritative. MiniMax-H3 does not get a weaker definition because it is a flow model.

For every strict reduction-bearing boundary:

1. accumulator precision is declared and fixed;
2. reduction and addition order is independent of batch size, batch position, packed-row count, launch geometry, SM/CU count, runtime occupancy, checkpointing, and distributed rank placement;
3. Split-K, Stream-K, split-KV, and cross-CTA atomics are forbidden unless the split and merge tree is itself part of the contract;
4. TF32, fast-math reassociation, approximate transcendental substitutions, and compiler-dependent contraction are disabled in strict mode;
5. casts occur only at declared epilogue or checkpoint-compatible boundaries;
6. one batch row is invariant to unrelated batch rows and batch permutation;
7. a logically equivalent permutation of packed rows, together with the same permutation of positions and metadata, produces the corresponding output permutation;
8. TP/CP/SP implementations are byte-equal to the WS1 result for the same logical rows;
9. unsupported strict geometry fails closed rather than silently using a production/native path;
10. runtime provenance records requested backend, actual backend, kernel ID, schedule ID, split policy, accumulator dtype, topology, checkpoint revision, task family, shape fingerprint, and fallback state.

H3-specific arithmetic rules

• FP32 modules remain FP32. Their inputs are cast at the same point as the pinned reference implementation.
• adaln_indices, timestep_indices, token_tags, position_ids, and modality-selection indices are semantic inputs, not incidental metadata.
• Q/K RMSNorm happens before MM-RoPE.
• MM-RoPE constructs frequencies in FP32 and rotates exactly 96 channels.
• Full attention is non-causal and unmasked over one packed document in the released path.
• AdaLN projection activation runs at the timestep embedding precision, then casts for the BF16 projection.
• Residual gates are applied in the declared order: residual + gate[row] * sublayer_output.
• Video and audio schedulers have separate sigma grids and must not share a step index implicitly.
• Sigma construction, duplicate removal, timestep conversion, velocity sign, denoised estimate, and Euler blending are all part of the strict trajectory contract.

Comparison hierarchy

Level	Required comparison
Operator accuracy	candidate versus independent FP32 reference under the shared tolerance profile
Operator invariance	byte equality across batch placement, shape-preserving layout changes, and repeated execution
WS1 chain	byte-equal selected intermediate tensors and first-drift localization
WS2 chain	byte equality against WS1 for identical logical rows
One-step model	byte-equal video and audio velocity outputs
Full trajectory	byte-equal latent state after every scheduler step
End-to-end media	latent equality is the hard gate; decoded media hashes/metrics are recorded as secondary evidence

Platform acceptance:

• CUDA: exact strict-path equality within a pinned CUDA architecture/runtime profile.
• ROCm: exact strict-path equality within a pinned ROCm architecture/runtime profile.
• CUDA versus ROCm: byte equality is tested and reported but not assumed. Any unavoidable platform transcendental/compiler difference requires an explicit named comparator profile and provenance.

---

5. Kernel and work-item table

Status: OPEN -> IN PROGRESS -> IN REVIEW -> MERGED.

To claim a task, put your handle in the GitHub column and open a PR. A row is not complete until implementation, an independent reference, invariance tests, runtime provenance, and row-specific acceptance evidence land together.

Reuse legend:

• Reuse - existing Qwen3 arithmetic can be used after H3 shape and dtype qualification.
• Extend - infrastructure is reusable, but H3-specific semantics or shapes are required.
• New - a new operator, contract, or integration boundary is required.
• Contingent - blocked on an explicit design decision or unreleased upstream component.

Work item	What it does	Track	Platform	Reuse / dependency	GitHub	PR	Status
h3_arch_fingerprint	Freeze revision, variants, 50-layer graph, refiner, mixed precision, scheduler and component hashes	foundation	CUDA + ROCm	New			OPEN
h3_packed_layout_schema	Machine-readable row identity, index tensors, modality/timestep tags and legal permutations	foundation	CPU	New			OPEN
h3_operator_trace	Per-layer requested/actual operator trace consumed by WS1/WS2 gates	foundation	CUDA + ROCm	Extend #315, #338, #360			OPEN
h3_trajectory_artifact	Seal initial noise, sigma grids, per-step velocities, latent hashes and first drift	foundation	CUDA + ROCm	Extend #230 artifact discipline			OPEN
video_patch_pack_unpack	(B,C,T,H,W) to (B,S,96) patch permutation and exact inverse	WS1	CUDA + ROCm	Extend #386 latent_pack_unpack			OPEN
video_input_projection_fp32	FP32 96 -> 5376 projection and backward	WS1	CUDA + ROCm	Extend #180/#343 deterministic GEMM			OPEN
audio_input_projection_fp32	FP32 32 -> 5376 projection and backward	WS1	CUDA + ROCm	Extend #180/#343			OPEN
text_context_projection	BF16 5120 -> 5376 projection and backward	WS1	CUDA + ROCm	Reuse #180/#343			OPEN
packed_index_copy	Deterministic zero-init and text/video/audio row scatter, including backward gather	WS1	CUDA + ROCm	New			OPEN
packed_index_select	Deterministic video/audio row selection and gradient scatter-add with fixed order	WS1	CUDA + ROCm	New			OPEN
timestep_sinusoid_h3	FP32 256-channel H3 timestep features with pinned sin/cos convention	WS1	CUDA + ROCm	Extend embedding infra			OPEN
timestep_mlp_fp32	FP32 256 -> 5376 -> 2688 timestep MLP	WS1	CUDA + ROCm	Extend #180 and elementwise kernels			OPEN
adaln_projection_3mod	SiLU + 2688 -> 96768 projection and six-way/three-modality table layout	WS1	CUDA + ROCm	Extend #180/#280; H3 layout is new			OPEN
adaln_row_gather	timestep_index * 3 + token_tag lookup for six modulation tensors	WS1	CUDA + ROCm	New			OPEN
h3_rmsnorm	Block, refiner and final RMSNorm with exact epsilon/cast order	WS1	CUDA + ROCm	Reuse existing RMSNorm infra			OPEN
h3_qk_rmsnorm_d128	Per-head Q/K RMSNorm for 56 heads and D=128	WS1	CUDA + ROCm	Extend Qwen QK-norm infra			OPEN
h3_qkv_gemm	Three deterministic 5376 -> 7168 projections	WS1	CUDA + ROCm	Extend #180/#343			OPEN
h3_mm_rope_3axis_partial	FP32 (t,h,w) frequencies and rotate-half on 96/128 channels	WS1	CUDA + ROCm	Extend #228; H3 partial layout is new			OPEN
h3_full_attention	Non-causal MHA, 56 heads, D=128, full packed document	WS1	CUDA + ROCm	Extend #240			OPEN
h3_attention_o_gemm	Deterministic 7168 -> 5376 output projection	WS1	CUDA + ROCm	Extend #180/#343			OPEN
adaln_gate_residual	Fixed x + gate[row] * y for attention and MLP branches	WS1	CUDA + ROCm	Extend #386			OPEN
h3_ffn_gate_up_gemm	Deterministic 5376 -> 2x14336 projection	WS1	CUDA + ROCm	Extend #180/#322/#343			OPEN
h3_swiglu	FP32-math SwiGLU forward/backward at H3 shapes	WS1	CUDA + ROCm	Reuse #280 after qualification			OPEN
h3_ffn_down_gemm	Deterministic 14336 -> 5376 projection	WS1	CUDA + ROCm	Extend #180/#322/#343			OPEN
token_refiner_block	Two pre-norm attention/FFN blocks without AdaLN or RoPE	WS1 chain	CUDA + ROCm	Compose qualified WS1 ops			OPEN
final_adaln_out	Final RMSNorm plus FP32 SiLU/projection shift-scale path indexed by timestep	WS1	CUDA + ROCm	Extend AdaLN rows above			OPEN
video_output_projection_fp32	FP32 5376 -> 96 projection and row selection	WS1	CUDA + ROCm	Extend #180/#343			OPEN
audio_output_projection_fp32	FP32 5376 -> 32 projection and row selection	WS1	CUDA + ROCm	Extend #180/#343			OPEN
h3_sigma_schedule	Shifted FP32 sigma grid, terminal zero, duplicate collapse and timestep conversion	WS1	CUDA + ROCm	Extend #386 flow_sigma_table			OPEN
h3_ode_step	Data-ward x0 and deterministic FP32 Euler blend for video/audio	WS1	CUDA + ROCm	New H3 sign/convention			OPEN
conditioning_noise_mix	Deterministic t*x0 + (1-t)*noise for conditioning anchors	WS1	CUDA + ROCm	New			OPEN
ws1_one_h3_block	One real block, forward/backward, all intermediates and first-drift report	WS1 closeout	CUDA + ROCm	Extend #315 chain harness			OPEN
ws1_full_h3_transformer	2-layer refiner + 50 blocks + both heads with real weights	WS1 closeout	CUDA + ROCm	Extend #315			OPEN
ws1_dual_trajectory	Full video/audio multi-step trajectory with per-step byte equality	WS1 closeout	CUDA + ROCm	New trajectory gate			OPEN
tp_attention_7168	Head/inner-dimension ownership for QKV and O projection	WS2	CUDA + ROCm	Extend Qwen3 TP and fixed collectives			OPEN
tp_ffn_14336	Column/row-parallel H3 FFN with fixed reductions	WS2	CUDA + ROCm	Extend #322/#343/#360			OPEN
tp_adaln_3mod	Shard 96768-wide AdaLN projection while preserving six tensors and modality rows	WS2	CUDA + ROCm	New H3 ownership contract			OPEN
cp_full_attention	Deterministic Ulysses/ring full attention over packed H3 sequence	WS2	CUDA + ROCm	Extend #240/#360			OPEN
cp_metadata_coshard	Co-shard positions, tags, timestep indices, RoPE tables and selection metadata	WS2	CUDA + ROCm	New			OPEN
cp_arbitrary_length	Strict behavior for sequence lengths not divisible by CP size	WS2	CUDA + ROCm	Extend runtime CP plans			OPEN
sp_norm_adaln	Sequence-parallel RMSNorm/AdaLN/gated residual acceptance	WS2	CUDA + ROCm	Extend Qwen3 WS2			OPEN
distributed_packed_io	Distributed input scatter and final video/audio gather without row-identity drift	WS2	CUDA + ROCm	New			OPEN
ws2_tp2_tp4_tp8	Exact TP2/TP4/TP8 equivalence against WS1	WS2 closeout	CUDA + ROCm	Extend #361			OPEN
ws2_tp4_cp2	Exact TP4/CP2 equivalence including metadata ownership	WS2 closeout	CUDA + ROCm	Extend #361/#377			OPEN
ws2_tp4_cp2_sp	Sequence-parallel acceptance on top of TP4/CP2	WS2 closeout	CUDA + ROCm	Extend #361/#377			OPEN
ws2_pipeline_parallel	Optional PP2/PP4 layer partition with exact activation handoff	WS2 follow-up	CUDA + ROCm	Extend distributed runtime			OPEN
h3_ablation_overlay	Preserve #230 A0-A7/C0-C2 and add H3-specific factors below	ablation	CUDA + ROCm	Extend #230			OPEN
diffusers_runtime_adapter_h3	Bind strict ops into the pinned Diffusers H3 transformer and scheduler	integration	CUDA + ROCm	Extend #338/#360 runtime pattern			OPEN
h3_checkpoint_mapper	HF/Diffusers weights to training runtime, including mixed-dtype and FL2VA/Ref2VA keys	integration	CUDA + ROCm	New			OPEN
h3_training_provider	Model-neutral structural context and strict operator injection for training	integration	CUDA + ROCm	Extend #352 provider pattern			OPEN
vllm_runtime_adapter_h3	Bind and read back actual strict H3 operators in pinned vLLM support	integration	CUDA + ROCm	Extend #338; verify upstream support first			OPEN
sglang_runtime_adapter_h3	Bind and read back actual strict H3 operators in pinned SGLang support	integration	CUDA + ROCm	Extend #338; verify upstream support first			OPEN
h3_runtime_modes	Strict/audit/auto/off with fail-closed strict behavior	integration	CUDA + ROCm	Reuse #361			OPEN
qwen3vl_layer50_contract	Tokenizer/special tokens, multimodal processor and exact layer-50 hidden-state extraction	Phase B	CUDA + ROCm	New; separate model family			OPEN
visual_vae_contract	Encode/decode, latent normalization, f16t4d24 layout and tiling	Phase B	CUDA + ROCm	Extend #386 latent infra			OPEN
audio_vae_contract	Stereo encode/decode and 40 Hz latent layout	Phase B	CUDA + ROCm	New			OPEN
fl2va_pipeline_contract	Zero/first/last/first-last frame preprocessing and packed-layout construction	Phase B	CUDA + ROCm	New			OPEN
ref2va_pipeline_contract	Reference image/video/audio preprocessing and transformer_ref mapping	Phase B	CUDA + ROCm	New			OPEN
phase_b_full_chain	End-to-end encoded inputs through decoded video/audio with first drift	Phase B	CUDA + ROCm	Depends on Phase A + encoders/VAEs			OPEN
stochastic_transition_contract	Declare SDE, noise law and exact transition logprob before implementing RL logp	Phase C	CUDA + ROCm	Contingent; #204/#336 methodology only			OPEN
h3_lora_gradient_gate	Base frozen, LoRA gradients finite/nonzero and invariant across train/rollout layout	Phase C	CUDA + ROCm	Extend training adapters			OPEN
final_e2e_diffusers_cuda	Pinned Diffusers versus RL-Kernel full trajectory and performance experiment	final	CUDA	Extend #377 methodology			OPEN
final_e2e_diffusers_rocm	Pinned Diffusers versus RL-Kernel full trajectory and performance experiment	final	ROCm	Extend ROCm closeout methodology			OPEN
final_distributed_h3_run	Multi-step TP4/CP2 run with sealed per-step trajectory evidence	final	CUDA + ROCm	Extend #377			OPEN
cuda_perf_closeout	Optimize strict path after correctness closeout	perf	CUDA	Extend #343 and existing perf workflow			OPEN
rocm_perf_closeout	Optimize strict path after correctness closeout	perf	ROCm	Extend ROCm strict kernels			OPEN

---

6. Ablation matrix

The matrix is a diagnostic taxonomy, not a Cartesian product and not a second runtime-knob surface. Each row changes one factor while all other replay inputs remain sealed. Reports must identify the first operator, layer, denoising step, packed row, and modality that drifts.

6.1 Base A0-A7/C0-C2 taxonomy from #230

The established attention taxonomy remains stable so evidence is comparable across model families. H3-specific factors are added as an overlay rather than renumbering the base rows.

Row	Factor	H3 interpretation	Expected result
A0	Strict shared replay	identical weights, packed rows, positions, timesteps, schedules and strict operators	PASS, byte-equal
A1	Position / RoPE	perturb one (t,h,w) coordinate or MM-RoPE convention	EXPECTED DRIFT at first affected attention layer
A2	Q/K preprocessing	perturb Q/K RMSNorm epsilon, cast, order or backend	EXPECTED DRIFT at Q/K norm or QK
A3	Mask / sequence boundary	split one packed document, inject a mask, or merge documents	EXPECTED DRIFT at attention
A4	TP/CP logical ownership	give a row/head/metadata item to the wrong rank	REJECTED as non-comparable
A5	KV-cache identity / layout	no KV cache exists in the released H3 full-attention path	N/A; fail if a runtime silently introduces cache semantics
A6	Precision / rounding policy	change FP32 module, accumulator, cast point, TF32 or fast math	EXPECTED DRIFT at changed boundary
A7	KV block plan / merge order	interpreted as CP attention block plan and merge order	EXPECTED DRIFT when order changes
C0	TP partition preserving head mapping	change TP degree while preserving logical head/channel ownership	PASS, byte-equal
C1	Sample alone vs packed batch row	same request alone and at different batch positions	PASS, byte-equal
C2	Prefill tail vs decode tail	no autoregressive cache handoff in released H3	N/A; replaced by H-C2 trajectory control below

6.2 H3-specific mismatch probes

Row	Single changed factor	Strict arm	Probe arm	Expected first-drift signature
H1	Packed row order	canonical order with aligned metadata	permute rows but leave one metadata tensor unpermuted	packed_index_copy or first MM-RoPE/AdaLN use
H2	Modality tag	correct video/text/audio tag	flip one row's tag	first block AdaLN modulation
H3	Timestep-table index	correct distinct-timestep lookup	offset one row's index	first block AdaLN modulation
H4	AdaLN table layout	[t0m0,t0m1,t0m2,...]	modality-major or six-chunk mismatch	first block AdaLN gather
H5	Partial MM-RoPE	rotate 96/128 channels	rotate all 128 or wrong axis concatenation	first layer Q/K after RoPE
H6	Q/K norm and RoPE order	Q/K norm then RoPE	RoPE then Q/K norm	first layer Q/K
H7	Mixed precision	declared FP32 input/time/output modules	cast one path to BF16 early	corresponding projection or timestep MLP
H8	Attention document	one non-causal full document	causal mask, padding mask or per-modality documents	first attention softmax/output
H9	Velocity sign	x0 = xt + sigma*v	subtract velocity	first scheduler step
H10	Timestep convention	t = 1 - sigma	t = sigma or scaled 0..1000	timestep embedding before first block
H11	Sigma grid	H3 shifted grid with terminal zero and dedup	generic flow-matching grid	scheduler construction or first differing step
H12	Dual scheduler ownership	video shift 12, audio shift 3	share one grid/index	first modality whose sigma differs
H13	Euler arithmetic	FP32 ratio/blend and declared cast	sample-dtype blend or reordered expression	scheduler step
H14	Output-row selection	correct video/audio indices	stale or locally sharded global index	output selection
H15	Conditioning augmentation	sealed noise and t*x0+(1-t)*noise	different seed/order/formula	input latent before transformer
H16	Variant identity	matching FL2VA or Ref2VA weights/layout	mix transformer or preprocessing variant	checkpoint mapping or packed input
H17	Distributed metadata	hidden rows and all metadata co-sharded	shard hidden state only	first local AdaLN/MM-RoPE use

6.3 Invariant controls

Row	Control	Required result
H-C0	Repeat identical one-step model call	bitwise zero drift
H-C1	Request alone vs same batch row	bitwise zero drift
H-C2	Same initial noise and complete video/audio trajectory	bitwise zero drift at every step
H-C3	Canonical packed order vs fully consistent row permutation	outputs equal after inverse permutation
H-C4	TP1 vs TP2/TP4/TP8	bitwise zero drift
H-C5	CP1 vs accepted CP topology	bitwise zero drift
H-C6	Gradient checkpointing off vs on	byte-equal forward and accepted gradient profile
H-C7	FL2VA/Ref2VA repeated preprocessing	identical packed-layout artifact and component hashes

6.4 Minimum experiment arms

Arm	Train/runtime operators	Scheduler	Purpose
P/P	pinned production/reference path	pinned production/reference	baseline behavior and performance
R/P	RL-Kernel training, production rollout	same sealed schedules	localize integration mismatch
P/R	production training, RL-Kernel rollout	same sealed schedules	localize integration mismatch
R/R	RL-Kernel strict on both sides	RL-Kernel strict on both sides	target exact consistency arm

For the deterministic public sampler, the primary reported quantities are velocity drift, latent drift, first-drift location, reward, and performance. Transition logprob/KL fields remain absent until stochastic_transition_contract is approved and implemented.

---

7. Acceptance gates

Per-operator gate

• Forward and backward match an independent FP32 reference under the declared profile.
• Batch position, batch size, repeated execution, and unrelated batch-row mutation do not change a logical row's bytes.
• Shape coverage includes short synthetic sequences, non-power-of-two sequence lengths, realistic 768p token layouts, and the largest supported task layout that fits the validation device.
• Empty, non-contiguous, mixed-device, wrong-dtype, wrong-index and unsupported-topology cases fail clearly.
• Requested and actual backend/kernel identities are persisted.

WS1 exit criteria

• All Phase A WS1 rows are merged for CUDA or an explicitly named platform subset.
• One real H3 block passes forward/backward and first-drift validation.
• The two-layer text refiner and full 50-layer transformer pass with real checkpoint weights.
• Video and audio velocity outputs are byte-equal between the two compared strict executions.
• A complete dual-scheduler trajectory is byte-equal after every model evaluation and Euler update.
• Gradient validation covers input latents, text conditions, all trainable model parameters in the selected training mode, and LoRA parameters where enabled.

WS2 exit criteria

• TP2, TP4 and TP8 are byte-equal to WS1 for accepted shapes.
• TP4/CP2 and TP4/CP2/SP preserve packed-row identity and all row metadata.
• Actual collective algorithm, rank order, topology and fallback state are read back from runtime.
• No production/native attention, GEMM, norm, activation, scatter/gather, or scheduler path appears in a strict trace.
• First-drift tooling works across rank boundaries and identifies the owning rank and logical row.

Phase B exit criteria

• Qwen3-VL layer-50 features, VisualVAE latents, AudioVAE latents, and packed-layout artifacts are reproducible under pinned preprocessing.
• FL2VA and Ref2VA each have at least one sealed end-to-end fixture.
• Latent trajectory equality is the hard gate. Decoded media hashes and quality metrics are recorded but do not replace latent evidence.

Final experiment gate

• Matched workload, initial noise, component revisions, scheduler grids, prompts/reference media and hardware topology.
• No missing or non-finite required metrics and no deleted failed steps.
• Per-step sealed artifacts include runtime provenance and latent hashes.
• Performance comparisons separate model time, scheduler time, communication, VAE/text encoding, and media decode.
• Claims distinguish single-seed trajectory evidence from multi-seed reward generalization.

---

8. Recommended claim order

For contributors, the recommended order is:

1. h3_arch_fingerprint, h3_packed_layout_schema, h3_operator_trace, h3_trajectory_artifact
2. video_patch_pack_unpack, input projections, packed_index_copy, packed_index_select
3. timestep_sinusoid_h3, timestep_mlp_fp32, adaln_projection_3mod, adaln_row_gather
4. h3_rmsnorm, h3_qk_rmsnorm_d128, h3_qkv_gemm
5. h3_mm_rope_3axis_partial, h3_full_attention, h3_attention_o_gemm
6. adaln_gate_residual, H3 FFN projection/SwiGLU rows
7. token_refiner_block, final_adaln_out, video/audio output projections
8. h3_sigma_schedule, h3_ode_step, conditioning_noise_mix
9. ws1_one_h3_block, ws1_full_h3_transformer, ws1_dual_trajectory
10. TP attention/FFN/AdaLN, CP attention, metadata co-sharding, SP
11. ws2_tp2_tp4_tp8, then ws2_tp4_cp2, then ws2_tp4_cp2_sp
12. h3_ablation_overlay and H1-H17 mismatch signatures
13. Diffusers adapter, checkpoint mapper, training provider, then vLLM/SGLang adapters
14. Phase B Qwen3-VL/VAE/FL2VA/Ref2VA rows
15. Final end-to-end experiments and performance closeout
16. stochastic_transition_contract only when an RL algorithm requires transition logprob and the sampler probability model is explicit

---

9. Contribution notes

• A work item is not complete until implementation and its validation contract land together.
• Do not claim "H3 attention done" with a generic D=128 kernel. H3 requires 56 heads, 7168 inner width, Q/K RMSNorm, partial 3-axis RoPE, non-causal full attention, and packed-row invariance.
• Do not claim "H3 AdaLN done" without the three-modality table layout, per-row timestep indices, six outputs, mixed-precision activation/projection boundary, and WS2 ownership.
• Do not use decoded pixels or audio as the first numerical comparator. Compare packed inputs, per-layer tensors, velocities, and latent trajectories first.
• Do not reuse language-model KV-cache or selected-token-logprob acceptance language for a path that has neither semantic object.
• Correctness precedes performance. Optimized sparse attention, fused AdaLN, CUDA Graph capture, sequence padding, and scheduler fusion land only after they reproduce the unfused strict graph.
• Full attention and future sparse attention are separate runtime identities and separate acceptance rows.
• FL2VA and Ref2VA are separate pipeline identities even when their transformer configs match.
• Strict mode fails closed. Audit mode may observe production fallbacks but must report them as fallbacks.
• Every evidence bundle records checkpoint, code revisions, model variant, scheduler shifts, sigma hashes, hardware, driver/runtime, topology, dtype policy, actual kernels and comparator profile.

PR deliverables

Each kernel/operator PR should include:

• Python reference, portable GPU implementation, and platform-specific implementation where applicable;
• binding, semantic registry entry, hardware gating, clean fallback policy, and strict fail-closed behavior;
• forward/backward correctness, invariance, invalid-input, and realistic-shape tests;
• benchmark with exact environment and actual backend readback;
• operator documentation describing reduction order, accumulator precision, cast points, split policy, and known unsupported shapes;
• a reproducible command and machine-readable evidence artifact.

If you are interested

Claim one row below and link the implementation PR. Closely coupled rows may be claimed together, but broad claims such as "all attention" or "all WS2" should be split into reviewable arithmetic, ownership, integration, and evidence PRs.