fix: report Vulkan expert residency in telemetry and brain map - #891
fix: report Vulkan expert residency in telemetry and brain map#891MasterCATZ wants to merge 1 commit into
Conversation
|
The diagnosis is right and I want this fixed: Vulkan residency really does live in a separate registry ( Two things stop it going in as it stands. 1. It reverts a published security advisory- # Liveness is always public; hardware/scheduler internals only when a
- # request is authed (or no key set), so a configured key isn't leaked
- # past a bare 200 to an unauthenticated probe. (#SEC-8)
- if self._is_authed():
- payload["scheduler"] = self.server.scheduler.snapshot()
- payload["kv_slots"] = self.server.kv_slotsand on if self._is_authed() and eng and getattr(eng, "emap", None): # (#SEC-8) hide routing telemetry unless authedBoth of those gates are advisory SEC-8, published with v1.5.0 three days ago and credited to its reporter. Opening them makes scheduler state, KV slot count, tier sizes and the full routing telemetry readable by anything that can reach the port, on a server the user may have bound beyond localhost. The problem you hit is real — the dashboard cannot poll a gated endpoint. But the direction is inverted: the dashboard should authenticate, not the endpoint should open. The page is served by the same process that holds the key; it can send it. If that turns out to be awkward in the current UI, say so and we will work out where the key should live, but silently undoing a published advisory to make a panel populate is not a trade we can make. 2. CI11 of 17 checks are failing. I have not dug into which, because the SEC-8 question has to be settled first and the answer changes the diff. What I would take todaySplit it. The The Thank you for the |
grep -rn "VK=1\|glslc\|vulkan" .github/workflows/
(nothing)
The Vulkan backend has never been compiled by any job in this repository.
Not built, not linked, not run. backend_vulkan.c and the four GLSL shaders
could be broken on dev right now and every check would stay green.
Three Vulkan PRs are open as this lands -- JustVugg#891, JustVugg#729, JustVugg#789 -- all written
by people without the hardware, and reviewed by reading the diff. JustVugg#892 and
JustVugg#887 are Vulkan bug reports from users whose cards nobody here has.
WHAT LAVAPIPE BUYS, AND WHAT IT DOES NOT
Mesa's software Vulkan runs on any x86 runner. Locally it produces:
[VK] VRAM pressure-proofing: memory_priority on, memory_budget on
[VK] ready: llvmpipe (LLVM 20.1.2, 256 bits), compute qfam 0, memtype 0,
fused gate+up, absorb attention
[VK] expert tier active: routed quantized experts on the GPU (budget 320)
So it proves: the .comp shaders compile to SPIR-V, the loader finds a
device, queue-family and memory-type selection works, the extension
negotiation works, and the expert tier initialises.
It proves NOTHING about performance -- it is a CPU rasteriser and will be
slower than the ordinary CPU path, so any tok/s measured under it is
meaningless. It also does not reproduce driver-specific behaviour: the
VK_EXT_memory_budget under-reporting on RADV RX 6000 in JustVugg#891 will not
appear here. Those still need @Limalski, @BranBushes, @MasterCATZ and
@krusherpt on real cards.
THE JOB
Two steps, both of which can fail independently:
1. `make colibri VK=1` plus an assertion that all four .spv exist and that
the binary actually links libvulkan. glslc can reject a .comp while
backend_vulkan.c compiles perfectly, so shader compilation is its own
failure mode and gets its own check.
2. Run against Lavapipe with a fabricated config.json -- enough to reach
coli_vk_init(), which runs before any weight is read. The run then fails
on the missing model, which is expected and ignored; what is asserted is
the [VK] banner.
VERIFIED IN BOTH DIRECTIONS
Passing, with the exact commands the job runs:
27532 shaders/attention_absorb.spv linkato a libvulkan OK
19120 shaders/qmatmul.spv PASS: [VK] ready
16924 shaders/qmatmul_gate_up.spv PASS: expert tier active
4180 shaders/rmsnorm.spv
Negative control, one shader removed:
[VK] cannot open shaders/qmatmul.spv
[VK] Vulkan backend unavailable (tried shaders/qmatmul.spv; ...)
PASS: the gate FAILS, as it must
A job that cannot fail is worse than no job, and this repo has shipped one
of those recently: JustVugg#868's release check was named "coli would not resolve
these next to itself" and asserted file existence, so it passed while the
launcher could not select the engine (JustVugg#879).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 participants
What
The web UI brain map and
/healthtelemetry showed no Vulkan VRAM usage even when Vulkan experts were resident on AMD. This happened because:emap_emit()only marked CUDA experts as VRAM tier; Vulkan fell through to disk.tiers_emit()/hwinfo_emit()counted onlyCOLI_CUDAGPU stats./expertswas auth-gated, so the UI could not poll the map directly.Why
Vulkan residency lives in a separate registry (
vk_reg_at), not inm->pin/m->ecache, so the existing tier-classification logic never saw it.Fix
c/telemetry.h: Vulkan-awaretiers_emit,hwinfo_emit, andemap_emit.c/colibri.c: setvk_eligible=1on registered Vulkan expert tensors.c/openai_server.py: make/expertspublic.VRAM budget note
VK_EXT_memory_budgetunder-reports device-local budget on RX 6000-series RADV. The existingCOLI_VK_IGNORE_BUDGET=1override is unchanged.COLI_VK_RESERVE_GBis still user-settable and remains the real reserve limit.AI-generated
This PR was generated with AI assistance.