feat(dflash): registry-declared draft sidecars — pull fetches, auto uses, on requires - #686
feat(dflash): registry-declared draft sidecars — pull fetches, auto uses, on requires#686Kaden-Schutt wants to merge 39 commits into
Conversation
Session evidence (serve path, growing context) — and two fixes it forcedRan the committed 8-turn coding session ( DFlash arm — dies at turn 3: AR arm — all 8 turns, recall 3/3: KV is 68 KiB/token here (16 KV layers × 8 heads × 2 × 272 B), so 13k ctx is 0.9 GB; the card is tight because of 18.7 GB weights + ~5 GB fixed non-weight residency. Adding a 1.7 GB draft leaves nothing for VMM to grow into, and the failure is silent to the user (empty turns), which is exactly the audit's Broken 2 (draft VRAM charged nowhere). Consequences for this PR:
Two findings that are not this PR's:
|
hw-gate sol prelimsummary: Adds registry-declared DFlash draft sidecars: pulling a target also downloads its draft, run/bench/serve resolve that sidecar for auto/on modes, explicit draft paths retain precedence, on fails closed when the draft is absent or unloadable, and rm preserves a shared draft while another installed target declares it. It also tightens path-to-registry identity using canonical installed paths and drains GPU state after a failed Qwen3.5-family load rollback. run_hardware: true routes:
unavailable_routes: (none) claim_assessment: The author claims target pulls acquire registry-declared drafts, auto uses an available draft, on fails closed when missing or unloadable, explicit drafts win, off strips drafts, shared-sidecar removal is safe, and real qwen3.8 DFlash generation improves over AR without incoherence. Proving this requires real-fixture loads showing the intended draft in loader output, coherent decoded battery turns, tag/path parity for the XT fixture, and a failed on-mode load followed by a clean valid load; the supplied benchmark and unit-test reports are claims rather than gate evidence. |
hw-gate evidence — 2 lane(s) — verdict passlane hiptrx (gfx1201)hw-gate evidence
fixturesqwen3.6:27bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 31.4 status pass
qwen3.6:27b battery turn 0qwen3.6:27b battery turn 1qwen3.6:27b battery turn 2qwen3.6:27b battery turn 3qwen3.6:27b battery turn 4ornith-1.5:35b-a3b-mq4rsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 32.7 status pass
ornith-1.5:35b-a3b-mq4r battery turn 0ornith-1.5:35b-a3b-mq4r battery turn 1ornith-1.5:35b-a3b-mq4r battery turn 2ornith-1.5:35b-a3b-mq4r battery turn 3ornith-1.5:35b-a3b-mq4r battery turn 4lfm2.5:1.2bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 8.0 status pass
lfm2.5:1.2b battery turn 0lfm2.5:1.2b battery turn 1lfm2.5:1.2b battery turn 2lfm2.5:1.2b battery turn 3lfm2.5:1.2b battery turn 4qwen3.8:27b-mq4-xtsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 12.3 status pass
qwen3.8:27b-mq4-xt battery turn 0qwen3.8:27b-mq4-xt battery turn 1qwen3.8:27b-mq4-xt battery turn 2qwen3.8:27b-mq4-xt battery turn 3qwen3.8:27b-mq4-xt battery turn 4kernelnot run lane hipx (gfx1100)hw-gate evidence
fixturesqwen3.6:27bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 28.5 status pass
qwen3.6:27b battery turn 0qwen3.6:27b battery turn 1qwen3.6:27b battery turn 2qwen3.6:27b battery turn 3qwen3.6:27b battery turn 4ornith-1.5:35b-a3b-mq4rsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 33.6 status pass
ornith-1.5:35b-a3b-mq4r battery turn 0ornith-1.5:35b-a3b-mq4r battery turn 1ornith-1.5:35b-a3b-mq4r battery turn 2ornith-1.5:35b-a3b-mq4r battery turn 3ornith-1.5:35b-a3b-mq4r battery turn 4lfm2.5:1.2bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 8.1 status pass
lfm2.5:1.2b battery turn 0lfm2.5:1.2b battery turn 1lfm2.5:1.2b battery turn 2lfm2.5:1.2b battery turn 3lfm2.5:1.2b battery turn 4qwen3.8:27b-mq4-xtsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 18.9 status pass
qwen3.8:27b-mq4-xt battery turn 0qwen3.8:27b-mq4-xt battery turn 1qwen3.8:27b-mq4-xt battery turn 2qwen3.8:27b-mq4-xt battery turn 3qwen3.8:27b-mq4-xt battery turn 4kernelnot run |
… a seat First live run of the rung (workflow_dispatch on warpfront#686, run 33848923893) never reached Sol: review.py launched omp with cwd=checkout AND `--cwd checkout`, and the workflow passes `--checkout pr` relative to the job workspace, so omp resolved `pr/pr` and exited 1 before reading the diff. prelim.json came back null, hardware was skipped, and the status went red with "sol prelim unavailable". The dry runs on warpfront#679 did not catch it because the fake omp ignored `--cwd` and every test passed an absolute checkout. - review.py: abspath the checkout once after parse_args; all three seat launches and the git helpers inherit it. - tests/fake_omp.py: chdir to `--cwd` like the real binary, exit 1 with the same error shape when it does not exist. - tests/test_review.py: prelim with a relative --checkout from the parent directory must reach the seat and pass an absolute --cwd. Fails on the old script with the runner's exact error; passes with the fix. 103/103 in scripts/hw-gate/tests.
hw-gate sol verdict{
"claim_verdict": "not-exercised",
"confidence": 0.97,
"coverage": {
"gaps": [
"The batteries do not show DFlash loader logs, tau, or other evidence that a registry-declared draft was loaded; qwen3.6:27b and qwen3.8:27b-mq4-xt decode rates are consistent with AR.",
"No tag-versus-installed-path or symlink parity route exercised registry_entry_for_path.",
"No dflash_mode=on missing or unloadable-draft failure followed by a valid load exercised rollback_unfinished_qwen35 and GPU-pool recovery.",
"Target pull, shared-sidecar rm behavior, explicit-draft precedence, final off stripping, and registry sidecar digest/download behavior were not exercised by hardware evidence.",
"registry/models.json and registry/v1.json are policy files and require human review."
],
"surfaces_evidenced": [
"load",
"serve"
],
"surfaces_touched": [
"load",
"serve",
"config",
"registry",
"filesystem",
"docs"
]
},
"decision": "needs-human",
"eyeball": [
"All five qwen3.6:27b battery responses on gfx1201 and gfx1100 are coherent, answer their prompts, contain the expected substrings, and show no attractor or special-token leakage.",
"All five qwen3.8:27b-mq4-xt responses on both lanes are coherent and satisfy their prompts, but the evidence does not establish that its shared DFlash sidecar was active.",
"The unrelated ornith-1.5:35b-a3b-mq4r and lfm2.5:1.2b batteries are coherent but do not cover the changed DFlash registry behavior."
],
"phase": "verdict",
"rationale": "The hardware job passed every fixture on gfx1201 and gfx1100 with coherent decoded text, so it reveals no regression. However, the central behavior in crates/hipfire-cli/src/main.rs:2661, crates/hipfire-loader/src/lib.rs:1975, and registry/models.json was not demonstrated: there is no evidence of a registry sidecar loading, on-mode failure and rollback, path/symlink identity parity, pull, or rm semantics. Because the core claim remains unexercised and the PR changes registry policy files, the result requires human review rather than a greenlight.",
"regressions": []
}Floor: hard=['policy_paths: registry/models.json,registry/v1.json'] soft=["coverage_gaps: ['The batteries do not show DFlash loader logs, tau, or other evidence that a registry-declared draft was loaded; qwen3.6:27b and qwen3.8:27b-mq4-xt decode rates are consistent with AR.', 'No tag-versus-installed-path or symlink parity route exercised registry_entry_for_path.', 'No dflash_mode=on missing or unloadable-draft failure followed by a valid load exercised rollback_unfinished_qwen35 and GPU-pool recovery.', 'Target pull, shared-sidecar rm behavior, explicit-draft precedence, final off stripping, and registry sidecar digest/download behavior were not exercised by hardware evidence.', 'registry/models.json and registry/v1.json are policy files and require human review.']", 'model needs-human'] model_decision=needs-human final=needs-human |
![hipfire-sol[bot]](https://avatars.githubusercontent.com/u/308720658?s=60&v=4){kind=small}
There was a problem hiding this comment.
hw-gate sol verdict block: hw-gate.json reports failure for both hiptrx and hipx, null hipfire and daemon MD5s, and no fixture results. Consequently, the sidecar resolution at crates/hipfire-cli/src/main.rs:2590, path-form registry policy at crates/hipfire-registry/src/lib.rs:424, and required-draft rollback at crates/hipfire-loader/src/lib.rs:1965 were not exercised. Author-reported tests and benchmarks are claims rather than gate evidence; the failed hardware run requires block.
There was a problem hiding this comment.
hw-gate sol verdict block: The clean hipx batteries show no ordinary-load or generation regression, but all four hiptrx routes failed with exit 1 and no rows. The decision rules require block on any fixture failure. More importantly, the evidence never enabled DFlash: it therefore does not validate sidecar resolution in crates/hipfire-cli/src/main.rs:2590, pull inclusion at :1618, daemon policy lowering in crates/hipfire-daemon/src/main.rs:1338, or fail-closed draft loading in crates/hipfire-loader/src/lib.rs:1965. The author's central pull/auto/on/override and throughput claims remain unexercised.
There was a problem hiding this comment.
hw-gate sol verdict needs-human: The generic load changes did not regress the tested fixtures: qwen3.6:27b, qwen3.8:27b-mq4-xt, ornith-1.5:35b-a3b-mq4r, and lfm2.5:1.2b passed coherent batteries on gfx1201 and gfx1100. However, the central behavior added at crates/hipfire-cli/src/main.rs:2598 and crates/hipfire-loader/src/lib.rs:1965 was not observed: the evidence contains no proof that a registry DFlash sidecar loaded, no missing-sidecar on/auto cases, no pull flow, and no absolute-path resolution case for crates/hipfire-registry/src/lib.rs:424. Because registry/models.json and registry/v1.json are policy files and material coverage gaps remain, human review is required.
|
announcement: Holding on the policy floor only: registry/models.json and registry/v1.json changed, and that needs a human to accept the sidecar declarations (and to note that a beta build still fetches master's registry, so the field is inert on staging until it lands there). On hardware the change does what it says: a bare qwen3.6:27b tag under on/auto loads qwen36-27b-dflash-mq4.hfq and decodes coherently with tau up to ~9.7 where the same run on master is plain AR; the symlinked qwen3.8-27b.mq4-xt requested by path gets its entry and sidecar; on with the sidecar missing exits before load, auto warns and runs AR, and an unloadable draft under on is refused with the target's 15 GB actually returned (32404 MB free reported by the daemon, lfm2.5 loads fine on the same process afterwards). One nit: investigation: | question | route | result | evidence | |---|---|---|---| | Does dflash_mode=on with a registry TAG and no draft flag/env resolve the declared sidecar and actually run DFlash coherently (Sol gap 1)? | HIPFIRE_NO_REGISTRY_FETCH=1 HIP_VISIBLE_DEVICES=0 HIPFIRE_CLI_BIN=$HW_GATE_BIN/hipfire HIPFIRE_DAEMON_BIN=$HW_GATE_BIN/daemon python3 scripts/serve_harness.py --model qwen3.6:27b --tag qwen3.6:27b --mode battery --prompts-file benchmarks/prompts/hw-gate/serve-battery.json --max-tokens 256 --thinking off --thinking-effort none --max-think-tokens 0 --dflash on --port 18400 --home fable-home/pr-qwen36-tag-dflash-on --out $HW_GATE_EVIDENCE/pr-qwen36-tag-dflash-on.json --serve-log $HW_GATE_EVIDENCE/pr-qwen36-tag-dflash-on.serve.log (exact recipe: $HW_GATE_EVIDENCE/route.sh) | serve.log: | Does a PATH-form request for the symlinked install (qwen3.8-27b.mq4-xt -> ~/qcal/...) get registry identity via canonical comparison, resolve the (also symlinked) shared MQ4 sidecar, and keep /health.model as the requested path (Sol gap 2; serve/mod.rs:1216)? | same harness recipe on GPU 1, PR binary, --model /home/kaden/.hipfire/models/qwen3.8-27b.mq4-xt --tag qwen3.8:27b-mq4-xt --dflash on --port 18401 | serve.log: | A/B against master: same tag, same config (dflash=auto) — does master run AR while the PR picks up the sidecar, with identical coherence? | HIPFIRE_KERNEL_CACHE=fable-home/kcache-base route.sh base-qwen36-tag-dflash-auto-gpu3b 3 $HW_GATE_BASE_BIN qwen3.6:27b qwen3.6:27b auto ; then HIPFIRE_KERNEL_CACHE=fable-home/kcache-pr route.sh pr-qwen36-tag-dflash-auto-gpu3b 3 $HW_GATE_BIN qwen3.6:27b qwen3.6:27b auto | master: no draft line, tau=None on all 5 turns, decode 32.2–36.3 tok/s (AR), 5/5 coherent. PR: | Does dflash_mode=on fail closed BEFORE any load when the declared sidecar is not on disk (main.rs:2724)? | HIPFIRE_NO_REGISTRY_FETCH=1 HIPFIRE_HOME=fable-home/nodraft-home HIPFIRE_MODELS_DIR=fable-home/nodraft-models (contains only a HARDLINK of qwen3.6-27b.mq4, inode 790866, no draft) HIP_VISIBLE_DEVICES=3 $HW_GATE_BIN/hipfire run qwen3.6:27b --spec dflash -n 8 'Say hi' | exit=1 in 1.6 s, no daemon load: | Does dflash_mode=auto with the sidecar absent warn once and run AR with a coherent answer (main.rs:2729)? | same nodraft-models dir, config [speculation] mode=auto dflash=auto ($HW_GATE_EVIDENCE/auto-missing-sidecar.config.toml), HIPFIRE_KERNEL_CACHE=fable-home/kcache-d1, HIP_VISIBLE_DEVICES=1: $HW_GATE_BIN/hipfire run qwen3.6:27b -n 64 --no-stream 'What is the capital of France, and which river runs through it? Answer in one sentence.' | | Does dflash_mode=on with an UNLOADABLE explicit draft refuse the load in the daemon (loader lib.rs:1978), drain the target's VRAM (lib.rs:1700), not compound on the lazy retry, and leave the daemon able to load another model (Sol gap 3)? | PR binary, GPU 0, HOME=HIPFIRE_HOME=fable-home/d2-on-baddraft with [speculation] mode=dflash dflash=on and [developer] dflash_draft=/home/kaden/.hipfire/models/lfm2.5-1.2b.mq4 ($HW_GATE_EVIDENCE/on-unloadable-draft.config.toml): | Do the rm semantics hold on the real binary: foreign same-basename path removes only itself (round-1 regression), shared sidecar kept while a sibling declarer is on disk and removed with the last (main.rs:1774–1874)? | DUMMY files only, no model run: HIPFIRE_MODELS_DIR=fable-home/rm-scratch/models (qwen3.6-27b.mq4, qwen36-27b-dflash-mq4.hfq, qwen3.8-27b.mq4, qwen3.8-27b.mq4-pro, qwen38-27b-dflash-mq4.hfq) + elsewhere/qwen3.6-27b.mq4; | Failed experiments (recorded, not evidence against the PR): why did the first dflash=auto A/B and the first auto+missing attempt fail on BOTH binaries? | base-qwen36-tag-dflash-auto, pr-qwen36-tag-dflash-auto, -gpu3 (first attempt), first auto-missing run; probe pr-lfm25-gpu2-probe | unproven:
rationale: The hard floor fired on registry/models.json and registry/v1.json, so this is a hold regardless of what I found; what I found is that the code side is proven on hardware and clean. Every gap Sol listed is now closed with a decoded turn: (1) crates/hipfire-cli/src/main.rs:2668/2690 resolve_dflash_sidecar — a bare tag under on and auto loads hard floor: ['policy_paths: registry/models.json,registry/v1.json'] soft floor: ["coverage_gaps: ['The batteries do not show DFlash loader logs, tau, or other evidence that a registry-declared draft was loaded; qwen3.6:27b and qwen3.8:27b-mq4-xt decode rates are consistent with AR.', 'No tag-versus-installed-path or symlink parity route exercised registry_entry_for_path.', 'No dflash_mode=on missing or unloadable-draft failure followed by a valid load exercised rollback_unfinished_qwen35 and GPU-pool recovery.', 'Target pull, shared-sidecar rm behavior, explicit-draft precedence, final off stripping, and registry sidecar digest/download behavior were not exercised by hardware evidence.', 'registry/models.json and registry/v1.json are policy files and require human review.']", 'model needs-human'] |
hipfire-fable
Bot
added
the
needs-human
There was a problem hiding this comment.
The core claim holds on hardware: with no env var and no --model-draft, auto on qwen3.6:27b and on on qwen3.8:27b-mq4-xt both load the registry sidecar and every request runs drafter=dflash with coherent output, while master on the same route stays AR; on+missing, auto+missing, on+unloadable, auto+unloadable and explicit-draft precedence all behave as described, and the pull path fetched a sidecar with a matching sha256. Two things need fixing before this can land: the new fail-closed arm in finish_qwen35_load leaves about 5 GB of VRAM behind after a refused on-load (23.98 GB vs 18.81 GB with the same next model resident), and since serve keeps retrying lazily that compounds; and hipfire rm <variant> now deletes a DFlash draft that sibling targets and the standalone draft tag still own. The registry policy files also need the maintainer's eyes (deny_unknown_fields means pre-PR binaries will reject the new remote v1.json until they upgrade). Evidence is in the fable-evidence directory; decision is hold. Hard floor fired on registry/models.json + registry/v1.json, so the decision is hold regardless; what I did was close Sol's coverage gaps so the human review is about the policy files and two concrete defects, not about whether the feature works. The feature works: with the PR binary (hipfire 0183affc…, daemon b1b14059…) and no env var or --model-draft, dflash_mode=auto on qwen3.6:27b and dflash_mode=on on qwen3.8:27b-mq4-xt both loaded the registry-declared sidecar ('DFlash draft loaded: …qwen36-27b-dflash-mq4.hfq' / '…qwen38-27b-dflash-mq4.hfq'), every request logged drafter=dflash with tau 1.85–10.0, and all 20 decoded turns plus the hipfire run qwen3.6:27b --spec dflash output are correct and non-degenerate; the base binary on the identical route is AR (tau=None) — pos-auto-qwen3.6-27b-{pr,base}, pos-on-qwen3.8-27b-mq4-xt-pr, run-tag-spec-dflash-qwen3.6-27b-{pr,base}. The four claimed edge semantics are evidenced on hardware: on+missing bails with the pull hint (neg-on-missing…), auto+missing warns and runs AR (neg-auto-missing…), on+unloadable fails closed in the daemon (neg-on-unloadable…), auto+unloadable logs and falls back (neg-auto-unloadable…), explicit draft beats the sidecar (explicit-draft-wins…). The pull→sidecar→on-load chain was also exercised, unintentionally: my probe serve had no default_model, fell back to the schema default tag qwen3.5:9b, and pull_command fetched the target and its DFlash sidecar from Hugging Face with sha256s matching v1.json, then loaded the draft — that is a sandbox breach on my side (the host has network), disclosed in rollback-reuse-same-daemon-pr/README.txt; the 5.9 GB was deleted afterwards. Two things the author should fix before this lands: (1) the new fail-closed arm at crates/hipfire-loader/src/lib.rs:1968 leaves ~5.17 GB resident (vram-leak-summary.txt: 23.98 GB vs 18.81 GB with the same next model loaded), and because serve keeps 'serving lazily' each request retries the failing load, so a corrupt sidecar under on becomes a VRAM drain rather than a clean refusal — check whether rollback_unfinished_qwen35 frees token_embd/lm_head, and whether the mtp=on path shares the bug; (2) rm at main.rs:1783 deletes a draft that other pulled targets and the standalone draft tag still own (rm-shared-dflash-sidecar-probe.txt) — skip sidecars referenced by another present target or that are themselves a registry entry. Also worth the maintainer's attention: registry ModelEntry is deny_unknown_fields (lib.rs:177), so binaries older than this PR will fail to parse the remote v1.json once it carries dflash and drop to stale-cache/bundled with a warning — the same thing happened when sampling_profiles was added on 2026-07-22, so it is precedented, but it is the real cost of the policy change. A gate-side note, not a PR defect: base/scripts/hw-gate/run.py exports HIPFIRE_HOME into the harness env, and the CLI reads $HIPFIRE_HOME/config.toml in preference to the harness's /.hipfire/config.toml, so the harness's [speculation]/[memory] sections are silently ignored by mandatory routes (my attempt 1 in attempt1-hipfire-home-leak/ shows '--dflash auto' measuring dflash_mode=off); it is harmless while gate routes are dflash-off, but any future dflash route in run.py will measure defaults unless HIPFIRE_HOME is unset for the harness call.
…ll declares hw-gate Fable seat on #686 (hardware probe): `hipfire rm qwen3.8:27b-mq4-pro` deleted qwen38-27b-dflash-mq4.hfq while qwen3.8:27b and qwen3.8:27b-mq4-xt — both declaring that sidecar — were still on disk; those siblings then ran AR under dflash_mode=auto or refused under `on`. Same shape for the mq3 draft (3 targets) and the 9B mq4 draft (3 targets). rm now removes a declared sidecar only when no OTHER registry entry declaring the same file still has its own target present in the models dir; otherwise it prints `keeping DFlash sidecar <file>: still declared by <tags>`. rm_command is a thin wrapper over rm_with_registry(&RegistryV1) so the rule is unit-testable without env or network. Tests: rm_keeps_shared_dflash_sidecar_while_sibling_target_present, rm_removes_dflash_sidecar_with_last_declaring_target, rm_without_dflash_declaration_leaves_draft_file_alone.
github-actions
Bot
removed
the
needs-human
|
Both regressions the Fable seat found on hardware (run 33852962119) are fixed on this branch:
Measured on hipx (gfx1100, 24 GB), idle daemon after one refused
Fable's ~5 GB was the net after a second load reused same-sized pool buckets; with nothing loaded afterwards the full 19 GB is visible. Not addressed here (from the same verdict): |
There was a problem hiding this comment.
hw-gate sol verdict block: Block is mandatory because qwen3.8:27b-mq4-xt failed its selected battery. Independently, crates/hipfire-cli/src/main.rs:2716 breaks the documented HIPFIRE_DFLASH_DRAFT='' opt-out by permitting subsequent sidecar resolution. The central claim—automatic registry-sidecar attachment producing DFlash generation—was not exercised by the captured routes.
There was a problem hiding this comment.
The gate failure on qwen3.8:27b-mq4-xt is the host, not your PR: every spawn died at hipInit code 100 during the 09:08Z amdgpu/kfd outage and /dev/kfd is still refusing opens, so that battery needs a re-run once the host reboots. Sol's HIPFIRE_DFLASH_DRAFT='' regression does not hold — the daemon's own env read (hipfire-daemon/src/main.rs:1199-1209) still drops the draft on an empty value, so the opt-out is intact. What does block: the new basename-to-entry resolution in resolve_tag makes hipfire rm /elsewhere/qwen3.6-27b.mq4 (a different artifact that only shares the name) delete models/qwen3.6-27b.mq4.triattn.blended_v3.bin and models/qwen36-27b-dflash-mq4.hfq while the registered qwen3.6-27b.mq4 stays installed — master removes only the named file, and the y/N prompt shows just a count. Please only inherit an entry when the path canonicalizes to the models-dir artifact (or never sweep models-dir sidecars for an outside path) and print the delete set; the registry pairings also need a human read by floor rule, and the PR body's '20 targets' is stale (13 after 99c9c7f). Everything else checks out: your rm keeper fix works on the gate binary, all 18 new tests pass, every sidecar digest on this host matches v1.json, and the pre-outage hardware runs on this head's daemon show auto attaching the sidecar with drafter=dflash and on failing closed correctly. The floor already forces block/hold (registry/models.json + registry/v1.json changed; qwen3.8:27b-mq4-xt battery exit 1). I traced the battery failure to the host, not the PR: every spawn died at hipInit code 100 during the 09:08Z amdgpu/kfd outage, kfd is still EINVAL now, and the coherent rows hw-gate.json attributes to that fixture are from a stale 08:25Z JSON (pr686-r3-mandatory-fixture-failure-is-host-outage.txt). Sol's high regression at main.rs:2716 does not hold — the daemon's own HIPFIRE_DFLASH_DRAFT read (hipfire-daemon/src/main.rs:1199-1209, unchanged from master) drops the draft on '' regardless of params.draft, so the opt-out is intact end to end. Block stands on a regression I demonstrated on the gate binaries instead: the new basename-to-entry resolution (crates/hipfire-registry/src/lib.rs:430-438) makes hipfire rm <path> delete models-dir sidecars belonging to a still-installed registered model when the path merely shares the basename (pr686-r3-rm-basename-collision-collateral.txt; base removes only the named file), and the confirm prompt shows only a count. The feature itself is well evidenced: the prior round's pre-outage hardware runs used this head's exact daemon (b1b14059…) and a CLI that differs only by the rm-only commit def19e3 — auto attached the qwen3.6:27b sidecar (drafter=dflash, tau 6.82/1.85/3.25/9.67/5.00, five coherent turns; base ran AR), on-missing failed closed with the pull hint, auto-missing warned and ran AR, explicit draft won, unloadable sidecar under on rolled back — and this round def19e3's keeper logic works on the real binary (pr686-r3-rm-shared-dflash-sidecar-probe.txt), all 18 new/affected unit tests pass on this head (pr686-r3-cargo-test-*.log), every v1.json sidecar digest present on this host matches byte-for-byte, and the daemon ratchet is clean. What changes the decision: (1) restrict basename resolution so a path only inherits an entry when it canonicalizes to <models_dir>/<entry.file> (or have rm never add models-dir sidecars for a path outside the models dir) and list the delete set in the prompt; (2) a maintainer reads the 13 registry pairings; (3) the mandatory batteries re-run on a healthy host — nothing in this diff explains the exit 1.
github-actions
Bot
removed
the
hw-gate-blocked
There was a problem hiding this comment.
hw-gate sol verdict block: The gate policy requires block on any fixture failure. qwen3.8:27b-mq4-xt failed on gfx1201 while passing on gfx1100, so the cross-architecture load/serve route is not clean. Moreover, the batteries do not prove the central sidecar claim because they do not demonstrate a loaded DFlash draft or exercise the new pull, missing-sidecar, override, rm, and rollback paths.
There was a problem hiding this comment.
Blocking, but not for the reason the floor fired: the hiptrx qwen3.8:27b-mq4-xt failure is the harness attractor detector flagging a correct one-word compact-JSON answer under temperature-1.0 sampling with DFlash off and MTP drafting — PR and base give byte-identical turns with a pinned seed, so that fixture is clean. The block is the hipfire rm collateral reported last round and still present on e0c3bf0: because resolve_tag now matches any path by basename, hipfire rm /elsewhere/qwen3.6-27b.mq4 deletes the installed model's triattn and dflash sidecars from the models dir (master removes only the named file); please only resolve a path to a registry entry when it canonicalizes to <models_dir>/<entry.file> (or apply sidecar removal only when the removed path is the entry's own file), and note in the PR body that path-form loads now inherit tag policy (max_seq 262144/vmm for qwen3.8 where master used 32768). The good news is that everything else is now proven on hardware on this head: the pool drain makes a refused on load cost 0 bytes of retained VRAM against a clean load, auto pairs and drafts with the registry sidecar, on fails closed cleanly at both the CLI and the daemon and the daemon still serves the next valid load. Minor: the 'not pulled; running AR' line prints twice under hipfire run, and the registry JSON changes still need a maintainer read. The floor's evidence failure is not a PR regression: the hiptrx qwen3.8:27b-mq4-xt battery produced five correct turns and was failed by serve_harness.py:2037-2043 scoring a one-word compact-JSON answer as maxfreq=1.0 (r4-mandatory-exit1-is-detector-false-positive.txt); that run had dflash_mode=off and drafter=mtp, and with a pinned seed PR and base produce byte-identical turns and exit 0 both in the route.sh environment (r4-mandatory-ab-summary.txt) and in the gate's HIPFIRE_HOME-set environment where MTP loads exactly as in the gate run (r4-gatehome-ab-summary.txt). Sol's cited high regression at main.rs:1987 therefore does not exist. What does exist, demonstrated on the head binary against base, is the rm collateral: registry lib.rs:427-438 resolves any path by basename, and main.rs:1785-1828 then deletes the installed entry's triattn and dflash sidecars when asked to remove a same-named file outside the models dir (r4-rm-basename-collision-collateral.txt) — master removes only the named file. That is destructive to a working install and was already reported in round 3 (pr686-r3-rm-basename-collision-collateral.txt) without being addressed by the follow-up commit, so this is a block, not a hold. The same basename resolution also makes path-form loads inherit tag policy (max_seq 262144/vmm/sampling for qwen3.8:27b vs master's 32768 default; r4-gatehome-*.serve.log:75) — not shown to break anything, but unmentioned in the PR body and worth a deliberate decision. Everything else the PR claims is now evidenced on hardware on this head: e0c3bf0 removes the 5.17 GB retention after a refused dflash_mode=on load (per-process VRAM after fail+reload equals a clean control load to the byte, r4-leak-summary.txt); auto pairs and drafts with the registry sidecar (r4-pos-auto-qwen3.6-27b-pr: 'DFlash draft loaded', drafter=dflash, five coherent turns); on fails closed at the CLI with a pull hint when the sidecar is missing and at the daemon when it is unloadable, and the daemon then serves a valid reload with dflash tau 4.25; auto warns and runs AR when missing (the warning prints twice under hipfire run; AGENTS.md says one line). Registry policy files still require a human read regardless.
The default ~24-token prompt reports prefill_tok_s ~= 363 tok/s while a 4.4k-token prompt on the same binary reports 886: the short-prompt number is launch overhead, not prefill, and the JSON gave no way to tell. - Add --prompt-file <PATH>: prompt read verbatim (raw bytes, no trim), mutually exclusive with positional PROMPT words. - Standard-bench JSON gains top-level prompt_tokens (u64, as the daemon reports it in done.prompt_tokens), prompt_md5 (hex md5 of the exact prompt bytes), prompt_chars, and warnings[]; warn when prompt_tokens < 256 that prefill_tok_s measures launch overhead. - Same three values printed on the stderr banner next to model:/arch:. - No existing field renamed, nulled, or removed; default prompt bytes unchanged (audit 2026-09-02 fix slice: bench-prompt-evidence).
The daemon's done event has no prompt_tokens key; the prompt is prefill_tokens (rows prefilled) plus cached_tokens (prompt-cache prefix). Measured on a 7900 XTX the JSON reported prompt_tokens: null and never warned on the 24-token default prompt.
…roken 1) ModelEntry gains #[serde(default)] dflash: Option<Sidecar> alongside mtp/dspark; every dense qwen3.5:9b/27b, qwen3.6:27b and qwen3.8:27b quant-variant entry declares the draft file its <tag>-draft alias resolves to (same-bit draft where one exists, else the mq4 controller). hipfire pull/rm now fetch/remove the dflash sidecar like MTP/DSpark. registry_gen annotates dflash with sha256/size_bytes; v1.json was mirrored by hand (offline) to exactly what the generator would emit.
…flash Broken 1) load_params wires entry.dflash into params[draft] when dflash_mode is auto/on and no explicit draft was given (--model-draft, developer.dflash_draft, params[draft] all win): on without the pulled file bails with a 'hipfire pull <tag>' hint, auto logs one line and runs AR. run/bench re-resolve after the final CLI selector so config-off + --spec dflash still pairs; serve inherits it via load_params. Final off still strips the draft. The daemon lowers dflash_mode into SpecLoadCfg.dflash and the Qwen35 draft-load failure arm returns a load error (with bundle rollback, mirroring MTP) instead of AR fallback when on.
Replace the filename auto-match story (no such matcher exists) with the new behavior: pull fetches the draft sidecar, auto uses it when present, on requires it, developer.dflash_draft/--model-draft override. Point at the real 'DFlash draft loaded:' loader line instead of the two log lines that were never emitted.
The dflash SpecLoadCfg arm mirrors the mtp arm below it; one line of comment says so. 4154 <= 4155.
resolve_tag matched a bare file name against entry.file but not a path whose final component is that file name, so `serve --model ~/.hipfire/models/qwen3.8-27b.mq5` (what serve_harness passes) got entry=None and no dflash sidecar: measured on hipx as tau=None / 37 tok/s under --speculation dflash while the same tag resolved the draft.
…e canonical target find_model_path canonicalizes, so a target that is a symlink into another directory (every ladder artifact on the bench boxes) has a parent with no draft in it and the registry sidecar was never found. load_params now takes the models dir and the resolver searches it first, then beside the target. Regression test with a symlinked target. Measured: serve_harness --speculation dflash on qwen3.8-27b.mq5 ran AR (tau=None, 37 tok/s) before; the direct-tag probe resolved the draft (tau=14.2).
Resolving a path-form model to its registry entry changed /health.model from the requested path to the tag. serve_harness's warm probe compares health.model to the launched path by realpath, so it never saw the serve as warm and killed/respawned it every 180 s (measured: two spawn attempts, zero turns). Keep the entry lookup for sidecars and policy; name the served model the way it was requested.
Measured on a 7900 XTX (serve_harness session_coding, greedy, thinking off, q8 KV): qwen3.8:27b-mq5 + its mq5 draft completes turns 1-2 (tau 3.6/3.5) then dies at turn 3, ctx ~4.9k, with spec_step hipMemCreate out of memory; every later turn is an empty response. The same session under AR passes all 8 turns (13.4k ctx, 38.1 -> 34.4 tok/s). 18.7 GB weights + ~5 GB fixed residency + 1.7 GB draft leaves no room for KV growth. Drop the sidecar from the 27B mq5/mq6 tiers (and qwen3.5:27b-mq6); mq4-tier and below keep theirs (measured 202 tok/s on qwen3.8:27b).
…ll declares hw-gate Fable seat on #686 (hardware probe): `hipfire rm qwen3.8:27b-mq4-pro` deleted qwen38-27b-dflash-mq4.hfq while qwen3.8:27b and qwen3.8:27b-mq4-xt — both declaring that sidecar — were still on disk; those siblings then ran AR under dflash_mode=auto or refused under `on`. Same shape for the mq3 draft (3 targets) and the 9B mq4 draft (3 targets). rm now removes a declared sidecar only when no OTHER registry entry declaring the same file still has its own target present in the models dir; otherwise it prints `keeping DFlash sidecar <file>: still declared by <tags>`. rm_command is a thin wrapper over rm_with_registry(&RegistryV1) so the rule is unit-testable without env or network. Tests: rm_keeps_shared_dflash_sidecar_while_sibling_target_present, rm_removes_dflash_sidecar_with_last_declaring_target, rm_without_dflash_declaration_leaves_draft_file_alone.
hw-gate Fable seat on #686: a daemon that went through one refused dflash_mode=on load held ~5.17 GB more VRAM with the next model resident than a clean daemon, compounding under serve's lazy retry. Cause: free_qwen35_bundle returns every buffer to the Gpu pool (free_tensor has no size cap, dispatch.rs:3261) and only unload_model drains it (lib.rs:3871-3872); a load that fails in finish_qwen35_load never reaches unload_model, so the whole target stayed pooled and the next load reused only the same-sized buckets. rollback_unfinished_qwen35 now mirrors unload_model: invalidate_graph_state + drain_pool after the frees. Covers all three callers: CASK eviction failure, dflash_mode=on draft failure, and the pre-existing mtp=on head failure (same leak, older than this PR).
… artifact hw-gate Fable seat on #686 (run 33889233321): resolve_tag matched any path by its final component, so `hipfire rm /elsewhere/qwen3.6-27b.mq4` — a different file sharing the basename — resolved to the qwen3.6:27b entry and deleted the installed model's triattn and DFlash sidecars while the model itself stayed; a same-basename foreign file loaded by path inherited the entry's sidecar and kv/max_seq policy. registry: resolve_tag drops the file_name() arm (a path is not a tag); bare entry.file names still resolve; new entry_for_file for exact matches. cli: registry_entry_for_path(paths, registry, input) — a path-form input resolves only when canonicalize(input) == canonicalize(models_dir/entry.file) for some entry, so the -xt symlink into ~/qcal still matches by target and a lookalike elsewhere never does. rm, run, bench, and serve (incl. the pre-warm thread) route through it. dflash_mode=on on a path with no entry and no explicit draft now fails closed: "DFlash draft required (dflash_mode=on) but <path> is not a registry-managed artifact; pass developer.dflash_draft or use the registry tag" (auto still serves it as a bare artifact) — closes Fable's earlier note that `on` ran AR silently there. The daemon only consumes CLI-lowered params and needs no change. cargo test -p hipfire-registry: 21 passed (2 new); -p hipfire-cli: 220 passed (4 new).
876cf28 to
c86712d
Compare
github-actions
Bot
removed
the
needs-human
# Conflicts: # crates/hipfire-cli/map.md # crates/hipfire-runtime/map.md
…, pool drain on refused load, path-identity fail-closed) to staging Policy floor (registry/models.json, registry/v1.json) means the decide seat can never land this; merged by Main on the gate's hardware evidence after four rounds of Fable findings, all fixed and measured: - pool drain on refused load (d233d2a): 18,950 MB retained -> 281 MB, and the same daemon then serves qwen3.8+draft at -20 KB vs a clean daemon (hipx gfx1100) - shared-sidecar rm guard (def19e3): rm no longer deletes a sidecar another target still declares - path-basename identity (876cf28): registry_entry_for_path requires canonicalize equality, so a foreign /elsewhere/qwen3.6-27b.mq4 gets no sidecar and dflash_mode=on fails closed instead of running AR silently Enablement is unchanged: dflash_mode default stays off and the sidecar is resolved only under auto/on, so a paired draft on disk still never drafts until the user opts in. The only user-visible delta is pull size: +0.55 GB (9B) / +0.92-0.98 GB (27B). 249 tests pass in hipfire-registry + hipfire-cli at 397a366 (includes the beta merge with regenerated cli/runtime crate maps).
|
Merged to The gate could never land this itself: All four rounds of seat findings are in, each measured rather than argued:
249 tests pass in |
…iffs Two policy gaps this ladder exposed. 1. The gate never ran DFlash. #686 (draft sidecars), #691 (draft ctor rollback), #692 (primer replay) and #702 (dedicated verify kernels) all went through with every lane green while speculation never once executed. #692's DFlash-arm defect -- primer replay systematically missing the most recent assistant body -- was found only because a seat thought to drive twenty turns by hand. That is not a gate. The load bucket now runs `battery-dflash` and the serve bucket `chain-dflash`: the same prompts with `--dflash on` and an explicit `--draft`. `on` rather than `auto` because `auto` silently falls back to AR when the draft is missing, and a route that can pass without speculating proves nothing. The draft is named explicitly because the canonical xt trunk is a symlink out of the models dir, so the daemon's filename auto-match finds nothing and would run AR. `dflash_draft` is a candidate LIST because the lanes hold different drafts: hiptrx has qwen36-27b-dflash-mq4.hfq and no qwen38, hipx has qwen38-27b-dflash-mq4.hfq and no qwen36. A lane speculates with the first candidate it holds; a lane holding none records `skip`. `skip` is neither pass nor fail. The aggregation was `all(status == "pass")`, which would have counted a skip as a fixture failure -- a false negative on evidence the host never had -- while treating it as a pass would claim coverage that did not happen. Skips are recorded and reported, and a genuine failure alongside a skip still fails. Coverage is asymmetric until both hosts hold both drafts. Pulling qwen38-27b-dflash-mq4.hfq to hiptrx and qwen36-27b-dflash-mq4.hfq to hipx (0.92 GB each) makes it symmetric; that is a disk decision, so the evidence says `skip` rather than silently pulling. 2. Sol refused hardware for any diff touching a filesystem path, which caught #689 for adding `--prompt-file` to `hipfire bench` and cost that rung a lane until `hw-run` overrode it. hipfire is a CLI inference engine: users name models, prompts, drafts and sidecars at invocation, and the gate's own harness passes exactly those flags. sol.md now separates whose path it is -- an explicit argument is ordinary product work; credentials, dotfiles, SSH or cloud config, /proc or /sys beyond device enumeration, assembled traversal, or a read whose result leaves the process still warrant refusal. Tests: eight new cases in scripts/hw-gate/tests/test_run.py covering flag translation (battery-dflash -> `--mode battery --dflash on --draft ...`), plain battery never receiving a draft, per-lane draft selection, skip-not-fail with the harness never invoked, chain-dflash keeping its own prompts, the skip-vs-genuine-failure aggregation, and a manifest assertion that the buckets actually carry the routes. 113/113 hw-gate tests pass.
Second live run on warpfront#686 (33849478193): Sol authorized hardware, both lanes started, both failed inside 40 s with "[Errno 2] No such file or directory: 'cargo'". No GPU was ever touched. Both runners are systemd services. `actions-runner/.path` was written at install (2026-07-11) and is the distro default PATH; the service environment has no ~/.cargo/bin, no /opt/rocm/core/bin, no ROCM_PATH. The warpfront#679 evidence runs were driven by hand from a login shell, which is why this never surfaced. Nothing on either host changed. hw-run and fable-decide now import the runner user's login PATH and the ROCm variables (ROCM_PATH, HIP_PATH, HSA_PATH, HIP_PLATFORM, LD_LIBRARY_PATH) via GITHUB_PATH / GITHUB_ENV before the build steps. Verified under a bare service environment on hiptrx: the step yields ROCM_PATH=/opt/rocm/core, HIP_PATH, LD_LIBRARY_PATH, and cargo/hipcc/omp resolve. hipx's login shell carries the same variables.
…t's raw output on a no-decision Run 33866758629 (warpfront#702) ended the decide phase in 10 s with "omp decide: no JSON object in assistant text", and the uploaded fable-evidence/ contained warpfront#700's fable-summary.md and warpfront#686's route outputs: the workflow does `mkdir -p fable-evidence fable-home` in a reused runner workspace, so every session inherits the previous PR's files and can cite them as its own. `rm -rf` both before the mkdir. On the no-JSON path review.py discarded the assistant text it had already extracted, so the artifact carried nothing to diagnose the failure with. decision.json now records `fable_error` and `fable_raw` {assistant_text_tail, stderr_tail}; the step log gets the tail too. 103/103 in scripts/hw-gate/tests; workflow YAML parses.
Every rung of the 2026-09-04 ladder hit the same 409 on the staging merge: `crates/*/map.md` carries a `<!-- crate-map:generated -->` block that both branches regenerate, so any two PRs touching the same crate conflict there while their real code merges cleanly. warpfront#689, warpfront#690, warpfront#691, warpfront#686, warpfront#687, warpfront#688 and warpfront#682 all needed the same three manual steps -- merge staging in, regenerate the block with scripts/check-crate-maps.py, merge -- six of them tonight. A gate that decides merge-staging and then holds on a generated file is asking a human to run a script, which is not review. On a 409 the decide phase now retries locally: merge staging into the PR head, and if the conflicted set is generated maps only, re-run check-crate-maps.py for those crates, commit, and merge the result. The retry is deliberately narrow, because auto-resolving conflicts is exactly where a gate can do damage: - if ANY conflicted path is not a `map.md`, it declines and the hold stands with the offending paths named -- a real code conflict must reach a human - it regenerates rather than picking a side, so the committed block is what the tree actually generates, not whichever branch won - a failed regeneration, a git error, or a timeout all decline rather than force Test: `test_generated_map_retry_refuses_real_code_conflicts` builds a real repo with a conflicting `.rs` and asserts the retry returns no merge SHA and names the file. The guard is the part worth pinning; the happy path is exercised by the ladder itself. 122/122 hw-gate tests pass.
1 participant
Summary
DFlash was reachable only through an explicit draft path (
--model-draft,developer.dflash_draft/HIPFIRE_DFLASH_DRAFT).hipfire pull <tag>-draftset nothing,dflash_mode autoscanned nothing, and the daemon logged "DFlash disabled" only foroff— so every user following AGENTS.md / MODELS.md ("pull the draft, setauto, run") was on plain AR without knowing. The filename auto-matcher those docs describe does not exist in the tree.This PR makes DFlash drafts registry-declared sidecars, exactly like
mtp/dsparkonModelEntry:hipfire pull <target>fetches the draft;load_paramsresolves it forauto/onwhen no explicit draft is given;onfails closed when it is missing (CLI) or fails to load (daemon Qwen35 arm, mirroring MTP); explicit--model-draft/developer.dflash_draftstill win; a finaloffstill drops it. Docs rewritten to describe what the code does. Audit:docs/plans/audit-2026-09-02/audit-Dflash.md(PR #685), Broken 1.Which crate(s) does this touch?
crates/hipfire-registry,registry/models.json+v1.json(newdflashsidecar on 20 dense targets)crates/hipfire-cli(pull/rm sidecar handling;resolve_dflash_sidecarinload_params, re-resolved after therun/benchselector)SpecLoadCfg.dflash; Qwen35 draft-loadErrfails closed whenon)Evidence (RX 7900 XTX / gfx1100, HIP 7.15, hipx)
Same command, same registry tag, no env var and no
--model-draft, promptbenchmarks/prompts/humaneval_3_below_zero.txt(md537c5aad9f9efe93b5c47f27256bdf149),--runs 5 --warmups 3 --max-tokens 128 --backend noslots --workload stateless:--specc66abf3d8(gate cache)drafter=off)67d5f268cDFlash draft loaded: qwen38-27b-dflash-mq4.hfq, τ=10.5567d5f268cDecoded output eyeballed via
hipfire run --spec dflash --max-tokens 3000on the same prompt: correctbelow_zeroimplementation after the think span (437 tokens, τ=5.15 with thinking on). Not a token attractor.Registry note: the branch's bundled
registry/v1.jsoncarries the field; the CLI prefers the remotemaster/registry/v1.json(24 h cache), so the branch was measured withHIPFIRE_NO_REGISTRY_FETCH=1. After merge, users pick the field up on cache expiry or the nexthipfire pull.Artifacts:
qwen3.8-27b.mq4andqwen38-27b-dflash-mq4.hfq(sha256d0a74a232a…, 1209603072 B, matches the sidecar entry).bench JSON — this branch, --spec dflash
{ "protocol": "native-generate-v1", "model": "qwen3.8:27b", "loaded": { "type": "loaded", "arch": "qwen3_5", "dim": 5120, "layers": 64, "vocab": 248320, "vl": false, "reasoning_contract": "qwen_jinja", "reasoning_effort_native": true, "reasoning_efforts": [ "low", "medium", "xhigh" ], "cache_capable": true, "retry_reset_eligible": true, "continuous_batch_capable": false }, "gpu": { "type": "diag", "arch": "gfx1100", "hip_version": "7.15", "vram_free_mb": 6722, "vram_total_mb": 24560, "model_loaded": true, "model_arch": "qwen3_5", "kernels": 0, "kernel_hashes": 0 }, "vram_free_before_mb": 24522, "max_tokens": 128, "runs": 5, "batch": 1, "decode_tok_s": { "median": 202.6, "mean": 202.06, "min": 199.9, "max": 202.8, "stdev": 1.092886087385138 }, "prefill_tok_s": { "median": 386.3, "mean": 383.12, "min": 369.8, "max": 387.0, "stdev": 6.666153313568474 }, "wall_tok_s": { "median": 132.5, "mean": 131.92, "min": 129.4, "max": 132.7, "stdev": 1.263962024745993 }, "ttft_ms": { "median": 334.0, "mean": 336.84, "min": 333.3, "max": 348.9, "stdev": 6.036091450599459 }, "samples": { "decode": [ 199.9, 202.3, 202.6, 202.7, 202.8 ], "prefill": [ 369.8, 386.2, 386.3, 387.0, 386.3 ], "wall": [ 129.4, 132.4, 132.5, 132.7, 132.6 ], "ttft_ms": [ 348.9, 334.0, 334.0, 333.3, 334.0 ] } }bench JSON — master, --spec dflash (AR)
{ "protocol": "native-generate-v1", "model": "qwen3.8:27b", "loaded": { "type": "loaded", "arch": "qwen3_5", "dim": 5120, "layers": 64, "vocab": 248320, "vl": false, "reasoning_contract": "qwen_jinja", "reasoning_effort_native": true, "reasoning_efforts": [ "low", "medium", "xhigh" ], "cache_capable": true, "retry_reset_eligible": true, "continuous_batch_capable": false }, "gpu": { "type": "diag", "arch": "gfx1100", "hip_version": "7.15", "vram_free_mb": 8620, "vram_total_mb": 24560, "model_loaded": true, "model_arch": "qwen3_5", "kernels": 0, "kernel_hashes": 0 }, "vram_free_before_mb": 24522, "max_tokens": 128, "runs": 5, "batch": 1, "decode_tok_s": { "median": 49.9, "mean": 49.84, "min": 49.7, "max": 49.9, "stdev": 0.07999999999999864 }, "prefill_tok_s": { "median": 421.5, "mean": 420.43999999999994, "min": 411.8, "max": 424.6, "stdev": 4.5071498754756325 }, "wall_tok_s": { "median": 44.5, "mean": 44.5, "min": 44.3, "max": 44.6, "stdev": 0.10954451150103478 }, "ttft_ms": { "median": 306.1, "mean": 306.84000000000003, "min": 303.8, "max": 313.2, "stdev": 3.316986584235752 }, "samples": { "decode": [ 49.7, 49.8, 49.9, 49.9, 49.9 ], "prefill": [ 411.8, 421.5, 424.6, 423.3, 421.0 ], "wall": [ 44.3, 44.5, 44.6, 44.6, 44.5 ], "ttft_ms": [ 313.2, 306.1, 303.8, 304.7, 306.4 ] } }Test plan
cargo test -p hipfire-registry -p hipfire-cli— 19 + 212 pass, incl. newbundled_dflash_sidecars_name_pullable_filesand 5 sidecar-resolution tests (present /on+missing bails /auto+missing warns / explicit wins / finaloffstrips)runoutput read)serveroute is the one to run when it isNot in this PR