v0.3.1: beta → master — reconciled ladder, DFlash cache repair, admission, oom_guard, hw-gate 3.8 pin - #730
Draft
Kaden-Schutt wants to merge 207 commits into
Draft
v0.3.1: beta → master — reconciled ladder, DFlash cache repair, admission, oom_guard, hw-gate 3.8 pin#730Kaden-Schutt wants to merge 207 commits into
Kaden-Schutt wants to merge 207 commits into
Conversation
…down (G2) Classify a retained source once and decide one effective topology before any destructive side effect, so a refused load leaves the prior model usable (issue #666 G2). - admission::admit_source (read-only): open the source, classify arch_id + vision (tower-tensor decides; contract 179a20d), decide the effective topology (single/pp/ep), and refuse no-carrier / ambiguous / VMM allowlist / VMM+pp / carrier-pp / EP-arch / lfm2 vision-no-config / DFlash lm-head quant — all before any GPU/VMM/teardown work. - Carrier::admit_topology (default + qwen35/cohere2moe/maple/gemma4 overrides) mirrors each carrier's load-time pp refusal. - Split the load entries: path wrappers classify-then-load via load_admitted_with_gemma4_drafter / load_model_ep_admitted, which consume the retained SourceAdmission (no re-open, no re-classify). - Reorder the daemon load handler: daemon topology refusals + admission run BEFORE prior-model teardown; on refusal the prior model stays loaded. RATCHET-RAISE: daemon_lines 4155 -> 4176, traded for the ~21-line G2 source-aware admission block inserted before prior-model teardown. Verification: workspace build clean; full workspace test suite passes; qwen3.6:27b (fresh-daemon VMM commit) + qwen3.6-35b-a3b load and generate coherently; a bad-path load and a vmm+pp>1 load both refuse at admission with the prior model still generating afterward.
admit_source refused kv_backend=vmm for every EP load, so a DeepSeek V4 EP + vmm load that master serves (load_model_ep_with_kv_mode arch 9 arm) was refused at admission. Gate the refusal on matches!(arch_id, 5|6|10), mirroring master's per-arch dispatch, and keep the DS4 (9) arm vmm-capable. Also correct the no-reopen claim on the EP path: the per-arch EP loaders re-open path per rank, so the retained SourceAdmission.source is dropped there rather than consumed (single/pp route is unchanged).
Regenerate the hipfire-loader generated map block after the per-arch VMM refusal fix (admission.rs 274 -> 298 lines, 5 -> 6 tests; lib.rs 4879 -> 4881 lines). Keeps scripts/check-crate-maps.py --check green in CI.
The default ~24-token prompt reports prefill_tok_s ~= 363 tok/s while a 4.4k-token prompt on the same binary reports 886: the short-prompt number is launch overhead, not prefill, and the JSON gave no way to tell. - Add --prompt-file <PATH>: prompt read verbatim (raw bytes, no trim), mutually exclusive with positional PROMPT words. - Standard-bench JSON gains top-level prompt_tokens (u64, as the daemon reports it in done.prompt_tokens), prompt_md5 (hex md5 of the exact prompt bytes), prompt_chars, and warnings[]; warn when prompt_tokens < 256 that prefill_tok_s measures launch overhead. - Same three values printed on the stderr banner next to model:/arch:. - No existing field renamed, nulled, or removed; default prompt bytes unchanged (audit 2026-09-02 fix slice: bench-prompt-evidence).
The daemon's done event has no prompt_tokens key; the prompt is prefill_tokens (rows prefilled) plus cached_tokens (prompt-cache prefix). Measured on a 7900 XTX the JSON reported prompt_tokens: null and never warned on the 24-token default prompt.
…ropped hw-gate Fable seat on #689: the flag table lost its `--reasoning-on` row (base AGENTS.md:359) while the flag still exists in `hipfire bench --help`. Additive row only, as the PR body says.
… JSON, short-prompt prefill warning) to staging
llama::is_batchable_la admitted MQ4G256V2/MQ6/5/3/2G256V2 for WMMA
prefill only on gfx1200/gfx1201 while qwen35::is_batchable_la admitted
them on gfx11+gfx12 behind HIPFIRE_MQV2_GFX11_WMMA, so plain Llama/Qwen3
dense qt=44 models prefetched per-token on gfx1100/1151 while Qwen3.5/3.8
took WMMA — despite both doc-comments claiming an exact match (audit
2026-09-02 Broken 1).
Move the dtype set + arch set + kill-switch helper into
llama::mqv2_wmma_batchable / llama::mqv2_gfx11_wmma_enabled_from_env in
hipfire-runtime and delegate from both callers, so the lockstep is
structural. MQ4CG256 (qt=45) stays gfx12-only in both by intent.
Tests: rename the two gfx12-only llama admit tests to gfx11+gfx12
expectations, repoint qwen35 env-escape test at the shared helper, and
add mqv2_admit_llama_qwen35_lockstep asserting both gates agree over
the MQ-V2 dtypes x {gfx1100, gfx1151, gfx1201, gfx1030, gfx1010}.
…ections mq4v2_gemm_parity's Gaussian weights give both halves near-identical headers, so a wrong half-select hides in quantization noise despite the header comment promising a systematic blow-up (audit 2026-09-02 Missing 1). Add arm 2 using the disjoint-halves construction from mq4v2_residual_parity (half0 [-1,1], half1 [96,160]) over the same batch-size sweep: v2 output must match the f32 reference within 5% rel-RMS, and the swapped-headers negative control (as in mq4v2_moe_parity) must DISAGREE. Keep the Gaussian v1-vs-v2 arm and fix the header comment to state what each arm can and cannot detect. Add a host-side test proving the fixture discriminates with no GPU. Docs (audit Would-change 1-2): spec section 9 now records MoE as production-wired for qt=44, the XBATCH single-row path as ported, and the gfx11 kt+=2 / residual kt++ stepping; residual_mmq.hip loses its stale Experimental tag; the gfx12 QKV kernel loses its HYPOTHESIS/scaffold wording for the validated C-map statement.
DflashScratch::new_with_mq, new_windowed, DflashWeights::load, and build_generic_dflash_speculator could '?' out mid-construction, leaking earlier alloc_tensor results (GpuTensor/DeviceBuffer have no Drop). Record each allocation in a slot vec (gt!/wt!/at!) taken once into the final owner; the error arm frees completed layers plus staged slots. new_windowed frees the base scratch via alloc_or_free!; the generic builder frees weights (and scratch) on later failures. Success path is byte-identical. Mirrors load_dflash_state's or_free! (audit-Dflash Broken 3).
The make_spec_emitter Err exit ran after a successful spec.prefill without production_fail_closed_rollback_live, unlike every other post-prefill error exit. The target KV/DeltaNet/drafter hidden had advanced and host seq_pos/conversation_tokens were cleared, so the next turn could LCP against a dirty GPU. Route it through the same rollback + fail-closed error (audit-Dflash Broken 4).
…length generate_dflash fell back to AR only when prompt + max_tokens > ctx_capacity, but generate_spec hard-errors when prompt + max_tokens + block_size > ctx — requests in that band got gen_start followed by an error instead of the promised AR fallback. Both sites now share spec_ctx_request_fits (prompt + max + block <= cap). The mid-loop position + block_size >= ctx_capacity break now sets SpecRun::ctx_exhausted, which the qwen (v2 + legacy) and dense epilogues OR into the length decision: finish_reason=length with no cache store instead of a silent early stop (audit-Dflash Broken 5).
qwen_dflash_semantic_terminal_tests.rs carries historical rustfmt debt; CI enforces rustfmt on changed files, so adding one test there forces a 6k-line reformat. The new contract lives in qwen_dflash_ctx_exhausted_tests.rs and the debt file is restored to master byte-for-byte.
…efill_chunk has no V2 arms hw-gate Fable seat on #690 (run 33895641944), source trace verified: the shared MQ-V2 admit rule made llama::is_batchable_la admit plain Llama/Qwen3-dense qt44/47-50 artifacts to WMMA prefill on gfx11 and gfx12, but llama.rs::forward_prefill_chunk's per-layer matchers (qkv_is_mq ~:2570, wo_is_mq ~:3025, ffn_is_mq ~:3117, w_down_is_mq ~:3248) list only MQ4G256|MQ6G256|MQ3G256|MFP4G32 — an admitted V2 model skips the FWHT rotate and runs the V1 hfq4g256 launchers on V2 blobs: silently incoherent prefill. master's pre-existing mq4_v2_gfx12 arm had the same hole on gfx12; no gfx12 Llama-V2 artifact has ever tripped it. llama::is_batchable_la now refuses every *G256V2 dtype and MQ4CG256 on every arch, with the reason at the site. qwen35::is_batchable_la keeps the shared mqv2_wmma_batchable rule (its chunk path has the V2 arms; gfx11 kill-switch intact). llama_spec::batched_verify_eligible routes all seven weights through is_batchable_la, so it is covered without an edit. Lockstep test now asserts the true contract: agreement on every non-V2 dtype across 5 arches; for V2, qwen35 admits on gfx11/gfx12 and llama refuses everywhere. Spec §9 row and crate maps corrected. hipfire-runtime is_batchable_la: 7 passed; qwen35 is_batchable + lockstep: 9 passed.
…id-ladder failure frees all of them hw-gate Fable seat on #691 (run 33900101473): alloc_or_free! freed the base scratch on failure but the four already-allocated tensors were still locals with no Drop — a failure on the 2nd..5th alloc leaked k_full / v_full / k_cat / v_cat. They are now assigned into s as each succeeds, so the error arm's s.free_gpu covers the whole ladder. Same shape as new_with_mq's at!/live list.
…llama and qwen35; discriminating GEMM parity; spec §9) to staging
…e does not re-emit it Both jinja cache lookups (ar.rs, qwen.rs dflash) prepended the live turn's generation primer to every cached assistant body, on the assumption that the template renders history assistant turns bare. Qwen3.5's does; Qwen3.8's re-emits <think>\n\n</think>\n\n on history turns when thinking is off, so the spliced render carried the primer twice and the LCP died at the first assistant turn of every conversation: every turn re-prefilled from the last checkpoint (2-4k tokens/turn on the coding session; the '2 minutes to first token' complaint). Measured on a 7900 XTX, qwen3.8-27b.mq4, AR, HIPFIRE_QWEN_CACHE_TRACE=1: prior_past[26..] = ```python... rend_past[26..] = <think>\n\n</think>\n\n```python... lcp=26 prior_len=118 rendered_len=148 template_emits_history_primer probes the template with a one-exchange history and decides per template; both lookups use it.
Close v0.3.1 audit rows for shared DFlash rm keep, pull≠enable, ornith:fast aliases, RCCL_LIB, arch prefill defaults, memory.oom_guard, multi_slot_prefill_chunk, MTP flash inherit, and RepairForTerminal.
…cing The developer_var migration (02fd3c2) routed the HIPFIRE_LOG_FORMAT read in init_tracing() through the process snapshot. init_tracing() is the first thing main() calls, before the CLI has sent the process config, and developer_var -> active_or_local_process_config() get_or_inits the LOCAL fallback snapshot on first read. The real install_process_config then fails with "process configuration was already initialized" and every daemon launched through `hipfire serve` exits before listening. Read the ambient env there (as HIPFIRE_LOG beside it already does) and list HIPFIRE_LOG_FORMAT in check-env-docs' BOOTSTRAP_ENV so the next sweep does not re-route it. Reproduced on 20ba9ee: `hipfire serve ... lfm2.5-1.2b.mq4` -> FATAL. Fixed binary (daemon md5 8295faa2) listens and pre-warms. RATCHET-RAISE: daemon_lines 4205 -> 4208, traded for the three-line comment explaining why this one read must stay ambient.
Point current pull/smoke/acceptance examples at qwen3.8:27b-mq4-xt and the measured DFlash draft; keep small-model smokes for speed-only paths. Retire the remaining AGENTS coherence-gate acceptance stragglers.
…LS/multi-gpu, INDEX, AGENTS, skills)
…parity examples skip gfx11-only arms
This was referenced Sep 7, 2026
…r (24 GB cards OOM)
…enerate) onto beta
New crate hipfire-arch-diffusion, 25 HIP kernels, hipfire-quantize --flux-pipe
packer, `hipfire img`, daemon img_generate, POST /v1/images/{generations,edits},
registry flux.schnell:1 -> elphil/flux (Apache-2.0). Port changes on top of
philhug's branch: img_generate body moved to hipfire_generate::img so the
daemon stays arch-free (daemon_arch_refs 0); FLUX loads refused at admission
on non-gfx11 archs; HIPFIRE_* reads routed through developer_var; new deps
pinned in [workspace.dependencies].
Proof on hipx gfx1151 (Strix Halo, 96 GB unified), merge/imggen 6a137f9:
11/11 kernel parity examples; `hipfire img` seed 0 twice -> byte-identical
PNG (sha256 990b53d3...), 41.2 s cold / 17.0 s warm at 512x512 x 4 steps;
serve_harness --mode images: 200 with within- and cross-process byte parity,
all 8 fail-closed 400 cases. gfx1100 (24 GB): kernels pass, schnell OOMs at
the first activation alloc (documented). gfx1201: refused at admission with
"requires RDNA3/3.5 (gfx11 wave32 WMMA); detected gfx1201".
RATCHET-RAISE: daemon_lines 4208 -> 4303, traded for the img_generate / img_load dispatch arm and the image-route refusals in the daemon (body lives in hipfire_generate::img).
RATCHET-RAISE: bypass_total 247 -> 257, traded for the ten diffusion-path direct launches recorded in docs/governance/debt-dispatch-bypass.txt (hipfire-arch-diffusion row).
admit_source gains trailing vision param; SourceAdmission carries the validated vision_path; probe promotes tower-less trunks to VL and refuses tower-less / wrong-arch sidecars fail-closed. Qwen35Carrier loads the tower from a separately-opened sidecar HfqFile against the trunk vision_config (has_vision_encoder true after). Daemon wires params.vision plus HIPFIRE_VISION_SIDECAR (draft ladder semantics), arch-free. LoadCtx gains vision_path; legacy/EP entries pass None. Docs: ARCHITECTURE line + env-vars row; crate maps refreshed.
…on-only) Adds hipfire-quantize vision-only sidecar emission for qwen3.8-27b-vision.hfq: new lib module vision_sidecar (prefix/dtype policy + 333-name contract tests), --vision-only shorthand, F32 emission for vision norms/biases/pos-embed in the quant fallback, QUANTIZE.md recipe + hfq_dump census.
Every qwen3.8:27b* tier declares vision.file=qwen3.8-27b-vision.hfq (mmproj-style, sha/size TBD until the pack ships). Registry schema + validation, registry_gen vision kind, CLI pull/rm --vision plumbing with dflash-identical shared keepers, resolve_vision_sidecar (explicit > HIPFIRE_VISION_SIDECAR > registry models_dir > sibling stem-vision.hfq), docs, and vision pytest.
…only the tower A float32 from_pretrained of an 18-shard Qwen3.8-27B materializes ~110 GB of LM we never run and gets OOM-killed. The checkpoint is stored bf16, so loading bf16 and upcasting model.model.visual afterwards is lossless for the tower and needs ~2 GB.
…on every qwen3.8:27b tier Pack published to hipfire-models/qwen3.8-27b. The vision test now pins the shipped contract: every tier declares the same digest-pinned file.
RATCHET-RAISE: daemon_lines 4303 -> 4321, traded for the params.vision / HIPFIRE_VISION_SIDECAR override ladder in the daemon load handler (arch-free string plumbing; admission validates).
…idecar like dflash_mode
…'s off-gate can name it The CLI stripped params.vision under vision_mode=off, so an image request against the default config got the generic 'model has no vision encoder' — indistinguishable from a tower-less artifact. The daemon gate is the hard override anyway; it now records the sidecar it declined and the refusal says: vision_mode is off and the tower sidecar <path> was skipped; run `hipfire config set vision_mode auto` (or `on`) and reload.
…sidecar like dflash_mode GPU proof on gfx1201 (isolated HIPFIRE_HOME): default off -> no tower load, image request refused naming the knob and the skipped sidecar; auto -> tower loads from the registry sidecar, VL answer; on with the sidecar absent -> 'Vision tower qwen3.8-27b-vision.hfq is not pulled; run hipfire pull ... or pass --vision', load refused before allocation; on with it present -> loads. RATCHET-RAISE: daemon_lines 4321 -> 4398, traded for the vision_mode hard-override gate and the declined-sidecar refusal message in the daemon load/generate handlers.
…e VL parity floor
…peg-turbo); zune-jpeg dropped All vision carriers now decode JPEG through one shared helper, hipfire_runtime::imagedec (decode_rgb8/decode_dynamic + _path variants and a header-only probe_dimensions for the decompression-bomb guard). JPEG bytes (FF D8 SOI) go to libjpeg-turbo-rs decompress_to(.., PixelFormat::Rgb); anything else still goes through image::load_from_memory, so PNG alpha handling is untouched (dots.ocr keeps its RGBA-on-white composite via the preserved DynamicImage variant). Downstream smart_resize/CatmullRom/normalize/patchify are untouched. Parity: decode_rgb8(benchmarks/vision/images/doge.jpeg) is 537x529 with RGB-buffer sha256 45bb7423193c00359695e1c967676d86e82bd3f5d55aa1679a85df6c74a9cf55, identical to PIL/libjpeg-turbo (pinned in imagedec unit test). Dep notes: libjpeg-turbo-rs 0.8 with default features, so the SSE2/AVX2/NEON SIMD kernels stay on (scalar fallbacks otherwise). Per its README it is a pure-Rust libjpeg-turbo reimplementation (no C deps, no unsafe FFI), byte-for-byte cross-validated against C libjpeg-turbo in CI. The image workspace pin drops the jpeg feature; zune-jpeg + zune-core leave Cargo.lock (cargo tree -i zune-jpeg is empty). Callers migrated: qwen35-vl image.rs, dots-ocr image.rs, lfm2-vl image.rs, diffusion refimg.rs.
…bo); zune-jpeg dropped VL parity on doge.jpeg vs HF f32 reference: patches rel-L1 4.6e-3 -> 2.6e-3 (max 7.8e-3 = one u8 step), i.e. only the CatmullRom-vs-PIL-bicubic kernel residual remains; block_26 5.7e-2 -> 4.4e-2, post_merger 5.4e-2 -> 4.7e-2. All five committed JPEG fixtures decode with max|delta| 0 vs PIL/libjpeg-turbo.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v0.3.1 — beta → master
Draft. Do not merge. Blocked on the items in § Blockers.
What this promotes
166 commits on
betasince #586. The line is the reconciled ladder plus tonight's landings.Landed via the beta ladder (reconciled in 7b16762)
#682 source-aware admission (G2) · #686 registry-declared DFlash sidecars · #687 admission refusals (#683 family) · #688 free weight sidecars / Ep mesh axis · #689 bench prompt evidence · #690 MQ-V2 prefill admit rule · #691 transactional DFlash ctors · #692 template-aware primer splice · #700 retire change_gate · #701 overhaul S1+S2 · #708 gfx12 verify-graph default · gfx1100 DFlash launch-fusion series (S1–S8, #702 body) · residual split-K / ldsstage tiers.
Landed 2026-09-06 (ported onto beta, each validated on Qwen3.8-27B MQ4XT + draft, gfx1201)
cached=177→407; session-dflashcached≈ctxat 17K, ~150 ms prefill (was 9–30 s cold)HIPFIRE_RCCL_LIBoverride for split ROCm installsornith-1.5:fastalias → MQ4R SKUmemory.oom_guard(auto: APU on, dGPU off) — VRAM-budget check kept unconditional, only host headroom gatedmoe_topk_renorm_k8post-invalidation__syncthreadsHIPFIRE_*reads throughdeveloper_varcheck-env-docs.pyexit 0 (was 27 findings, pre-existing on master)--replay-route-proof-loginit_tracingreadHIPFIRE_LOG_FORMATvia the config snapshot before the CLI installed it (everyhipfire servefailed at startup on 20ba9ee)feat/wip-img-generatehipfire img,/v1/images/{generations,edits},--flux-pipepacker,flux.schnell:1→elphil/flux. Port: img body moved tohipfire_generate::img(daemon arch-free), gfx12 refused at admissionqwen3.8-27b-vision.hfq, 928 MB F16, mmproj-style): every text tier serves images without requantizing.--vision-onlypacker arm,visionregistry slot on all 16 qwen3.8:27b tiers (digest-pinned, on HF),pull/rmshared-keeper,--vision/HIPFIRE_VISION_SIDECAR, admission probes trunk-or-sidecar, carrier opens the sidecar as a separate HfqFile. Opt-in: `vision_mode = offKaden-Schutt/hipfire-site#site/v0.3.1check-env-docsclean, site buildsRatchets
RATCHET-RAISE: daemon_lines 4155 -> 4398(G2 admission block, sticky GPU-fault latch, DFlash sidecar plumbing, theinit_tracingcomment, img dispatch arm, vision sidecar param ladder + vision_mode gate),RATCHET-RAISE: bypass_total 237 -> 257(ten ledgeredgemm_*_f16launches in qwen35/prefill.rs; ten diffusion-path launches). Both declared in commit bodies; this PR carries theratchet-raiselabel.leanup-ratchets21/21,ratchet-diffvs master clean.Validation on the tip
cargo build --release --workspace --all-targets --locked·cargo test --lib --workspace --locked(2709 pass) · crate maps 43/43 ·check-env-docsclean · leanup 21/21 · ratchet-diff clean.9f91556f…) + draft (d0a74a23…): battery / chain / session, AR and DFlash — 0 attractor, 0 runaway, 0 empty; decoded text read./tmp/beta-val-7b16762b0/on the gate host (binary md5s recorded per run).Blockers (merge waits on all)
05908ea1fadmission refusal; needs re-proof on rebased tip)prefill.rs; G1–G6 re-run on qwen3.8feat/wip-img-generate— landed d09d73e, proven on Strix Halov1.json) does not carry thevisionslot until this merges, sopullonly fetches it via the bundled registry / after promotionagent://ReleaseDocsAudit); version bumpCargo.toml,CITATION.cff,CHANGELOG.md, README badgesClose on merge (superseded by this line)
#682 #686 #687 #688 #689 #690 #691 #692 #695 #700 #701 #708 #725 #726 #728 #729 #680 #697 · already closed: #702 #727 #685