fix #675: complete production Single manifest/load transaction (G3 redo, stacked on #682)

[fivetide]

Status — base beta (G2 landed on beta)

Base note: #682 (G2) content was merged into the beta branch (upstream beta now contains replan/g2-admission @ 95e0e654e); master does not carry it yet. This branch is rebased onto upstream/beta (10 commits, 6b5003daa → e7603508), so the diff against beta is the pure G3 change: +4564/−120 across 12 files. #682's GitHub PR on master remains open; its beta merge is what this PR stacks on.

Rebase onto beta required two shared-seam adaptations, both committed with rationale: (1) CollectiveHint::AllReduce restored in device_mesh.rs — beta commit 90b2cc7aa deleted it as unreferenced dead code, but the G3 manifest planner is the producer that makes it live (per-operation ordered collective schedule is part of the #666 contract); init_vram_weighted stays deleted. (2) Test-side LoadCtx::vision_path (beta vision-loader) plus a module Mutex serializing the pinned-fixture GPU evidence tests (hipMemGetInfo is device-global).

This is the replacement for the historical #675 draft (G3, displaced-stack provenance only). It does not close #666's G3 milestone by itself — milestone acceptance is recorded by the tracker JSON at the accepted composed head, per the wave doctrine.

2026-09-07 review preparation

Head 1e68f81818a40f0bd79983c854fab602ad040557 adds only a regenerated hipfire-arch-llama/map.md (carrier line count and total). python3 scripts/check-crate-maps.py --check passes for all 43 maps, and the new CI gates job passes. No Rust or runtime behavior changed. Hardware results below remain explicitly scoped to e7603508; this documentation-only commit is not a new hardware run or milestone acceptance. The separate advisory Clippy failure is in the Qwen35 example test_dflash_hidden_scatter_gfx1100.rs:29 (clippy::approx_constant), outside this G3 patch.

Summary

Implement the G3 manifest/store layer and wire the plain LLaMA Single HFQ production path to it, under one ArchModel owner. A plain LLaMA HFQ source is classified, admitted, and routed through pure manifest planning → transactional fulfillment → typed assembly → publication, with parity against the legacy loader proven bitwise on a pinned real fixture, and the full load lifecycle (reset smoke, unload, immediate reload, deterministic fault, immediate retry) evidenced on hardware.

What changed

• hipfire-runtime::weight_manifest — GPU- and source-format-free declaration layer. WeightEntry/StateEntry describe logical tensors and state; shard policies cover replication, tensor/vocabulary/head/expert placement, fused projections, ties, and mesh pins. plan_manifest resolves declarations against an admitted DeviceMesh into device placements, state residency, pipeline boundary hints, and an ordered per-operation collective schedule. Manifest validation checks identities, names, dtypes, shapes, divisibility, scope, ties, aliases, cycles, mesh epoch, logical rank, and physical device before fulfillment. Expert declarations are carried for the G5 consumer.
• hipfire-runtime::weight_store — keyed by (name, layer, device) with immutable projection metadata, resident or symbolic-alias handles, and a WeightOrigin carrying mesh epoch, logical rank, and physical device identity. fulfill_manifest validates the target/source contract, uploads residents, materializes ties, and on the first source/dtype/payload/upload/assembly failure rolls back every earlier resident in reverse order, no legacy fallback. Assembly guards restore taken cells when typed construction aborts; publication transfers handles only on finalization. Deterministic fault seam + resident accounting under #[doc(hidden)] pub mod test_support.
• hipfire-arch-llama — LlamaBundle::load_bundle classifies HFQ sources (HfqLoadRoute::{ManifestPlainLlama, LegacyAwq}), routes plain non-AWQ files through the manifest transaction and typed assembly, and retains AWQ-sidecar HFQ plus safetensors-directory sources on their existing loaders. Llama::weight_manifest{,_for_hfq} / state_manifest declare weights and KV state with source-dtype constraints, output/embedding pins, and conditional lm_head tying. The attached store is crate-private; the sole consuming owner is ArchModel::free_gpu (scratch → store drain → weights → KV), so teardown cannot encounter an origin mismatch or a second unload owner.
• Shared seams (minimal adaptation only): hfq.rs splits validate_llama_hfq_admission out ahead of every llama loader route; model_load.rs gains Layout::from_mesh + Layout::validate and an empty-devices guard; weight_backend.rs exposes the HFQ dtype bridge. No Architecture trait surface change.

Ownership

• G3 owns: one production Single manifest/load transaction and its publication, rollback, and unload drain under the LlamaBundle ArchModel owner, plus the committed pinned-fixture parity oracle and lifecycle evidence tests.
• Not G3: request/cache/session lifecycle (G4), typed UnloadError/retryable-preflight loader/daemon teardown machinery (G4 provenance from the old feat(runtime): add device-mesh weight manifest and store pilot #675 head is deliberately not ported), multi-device/non-Single fulfillment and MoE execution (G5), planner-side GPU/file I/O (the planner performs none).

Verification

All hardware evidence on halo (gfx1151, 128 GB UMA, ROCm/HIP 7.2), on the final composed beta head (e7603508), release binaries rebuilt from that head (hipfire md5 9d290b26c517b84b94053a37f287736d, daemon md5 3b68f139c2eab5a9366692f52d84f647).

Committed pinned-fixture parity oracle (pinned_fixture_manifest_legacy_parity_oracle)

Tracker fixture qwen3:0.6b = ~/.hipfire/models/qwen3-0.6b-llama.mq4, size 495,181,824, md5 2579e10ba3a988818386f2b07632ee01 (size + route class asserted in-test; fixture lock). Production ManifestPlainLlama route vs legacy load_weights_hfq reference decode the same committed prompt (md5 4f880dbf80dbff04c0690dd8dce80d39) greedily in lockstep:

• 19 committed positions, worst logit diff 0.000e0 (bitwise); token IDs, alias identity, KV geometry/byte extents, and position counters identical at every position; route identity recorded.
• End-state layer-0 K/V payloads byte-identical (69,632 bytes each, 0 byte diffs).

Committed lifecycle evidence (pinned_fixture_lifecycle_fault_retry_reload)

Same fixture: production load → decode → existing-reset smoke (reset_session_state, decode byte-identical) → unload via the sole ArchModel::free_gpu owner → immediate reload (decode byte-identical) → deterministic post-upload fault (rollback zero-free: resident allocations == releases) → immediate retry (decode byte-identical). Warm baseline: 311 resident store allocations. All legs decode [785, 6722, 315, 9625, …] identically.

Additional validation added after the first evidence pass

• AWQ-sidecar GPU load through the retained legacy route (production_awq_sidecar_loads_and_decodes_on_gpu_through_legacy_route): an AWQ-sidecar HFQ is classified LegacyAwq, loads on GPU through load_weights_hfq with no manifest store attached, and decodes.
• Repeated load/unload cycles (pinned_fixture_repeated_load_unload_cycles_leak_nothing): four production cycles on the pinned fixture publish exactly the warm baseline (311 residents) every cycle, decode byte-identically, and stay within a bounded free-VRAM floor. A comparison test on the legacy loader (pinned_fixture_legacy_route_cycles_vram_bounded) shows the same ~540 MB net per-cycle hipMemGetInfo drift on gfx1151 UMA (driver-pooled frees are not credited in-cycle, with async partial recovery) — the manifest route introduces no VRAM retention beyond the loader it replaces; the committed floor is 1 GiB/cycle so only a manifest-specific doubling trips it.
• Serving-seam reset smoke: hipfire serve pre-warms the pinned fixture (serve log: llama: HFQ source route = ManifestPlainLlama); two independent /v1/chat/completions turns (each a fresh engine session — the reset boundary at the serving seam) both returned correct decoded text ("The capital of France is Paris, and the capital of Japan is Tokyo." / "The largest planet is Jupiter, and its main moon is Europa.").
• Additional loader-path production run: hipfire run on the 15 GB qwen3.5-27b.mq4 (qwen3_5 arch — exercises the shared hfq/model_load seams at scale through its own carrier; coherent decoded text). Manifest-route scale is bounded by the pinned llama-family fixture per the G3 contract.

Loader-path production runs (decoded text read; Kaden's bar)

Command	Fixture (md5)	Result
hipfire run …/qwen3-0.6b-llama.mq4	2579e10b…	logs llama: HFQ source route = ManifestPlainLlama; coherent decoded text
hipfire run qwen3.6:27b	9a6acdc49bcaa6a7b52ac161444cb769	coherent haiku decoded
hipfire run …/qwen3.6-35b-a3b.mq4	edde51ec1dac0f2bd42cff5ef1cb8944	fluent correct paragraph decoded

Both big fixtures match the pinned identities in AGENTS.md. No perf numbers are claimed from these runs (fixture-digest discipline applies to perf reporting; none is made here).

Batteries (all green)

• hipfire-runtime lib: 628/628 on the beta head (incl. weight_manifest 7/7, weight_store 14/14 with live-GPU rollback/fault, hfq, model_load, weight_backend).
• hipfire-arch-llama lib: 18/18, stable across two consecutive full runs on the beta head (carrier manifest/legacy/alias/AWQ/fault tests + pinned-fixture oracle, lifecycle, AWQ-sidecar GPU legacy load, and load/unload cycle evidence; GPU evidence tests serialized).
• Scoped clippy (-p hipfire-runtime --lib, -p hipfire-arch-llama --lib): 0 errors.
• Scoped rustfmt on changed files: clean.
• scripts/leanup-ratchets.sh: 21/21 metrics, 0 violations; daemon_lines unchanged at the 4176 cap — not raised (historical feat(runtime): add device-mesh weight manifest and store pilot #675 raised it to 4589; this PR does not).
• Crate maps refreshed (check-crate-maps.py).

Nonclaims / retained routes

• Single-only by design. No physical multi-device manifest fulfillment, executor execution, or production non-Single route is claimed; those wait on accepted G2 admission meshes and G5.
• AWQ-sidecar HFQ and safetensors-directory sources remain on their established loaders until sidecar/directory ownership is represented by the manifest transaction. No Architecture trait surface change.
• No hipfire-hardware crate (Gpus stays in hipfire_runtime::multi_gpu); no Cargo.toml dependency added for the manifest/store/route work.
• The old feat(runtime): add device-mesh weight manifest and store pilot #675 typed-UnloadError/retryable-teardown loader/daemon machinery is not ported (G4 lifecycle ownership); it is retained as provenance for the G4 redo.
• No daemon, loader, or leanup-thresholds.txt edits in this PR.

Dependency and landing order

• Requires accepted G1 (hipfire_runtime::device_mesh, merged feat(runtime): add DeviceMesh topology (fixed #673 G1) #681) — consumed via the fallible coord API (Result<_, MeshError> with .expect on validated coordinates).
• Integration base is upstream beta, which already contains G2 head 95e0e654e59a3892777a6fc73894006a576c6e43. This PR is G3-only against beta; the still-open feat(loader): source-aware admission — classify and admit before teardown (fixed #674 G2) #682 targets master and does not need to merge there before this beta PR can be reviewed. G1/G2/G3 milestone acceptance still requires the agreed final-composition receipts; merge status is not acceptance.
• Historical feat(runtime): add device-mesh weight manifest and store pilot #675/fix(runtime): consolidate generation lifecycle ownership #676/feat(dispatch): seal collective-aware MoE step execution #677 heads are provenance only; never merge, rebase, or promote them.
• Revert identity: none applied — this PR contains no revert; it is a fresh implementation on the feat(loader): source-aware admission — classify and admit before teardown (fixed #674 G2) #682 head.