Skip to content

E0: measurement foundation (pin-load, timing hooks, lookup counters, gate) - #152

Open
ddvnguyen wants to merge 3 commits into
epic/148-per-expert-backend-selectionfrom
e0/measurement-foundation
Open

ddvnguyen wants to merge 3 commits into
epic/148-per-expert-backend-selectionfrom
e0/measurement-foundation

Conversation

@ddvnguyen

Copy link
Copy Markdown
Owner

E0 measurement foundation for epic #148, built fresh on the rebased epic base (86af0c9, contains upstream 90e0f5c). The gather pilot (PR #134) is NOT ported; only the concepts of #142/#143/#144 survive, per the epic ruling.

Closes #142, closes #143, closes #144. Dedupe substance of #146 absorbed into the #142 commit (range-check moves to the E1 validate hook).

Commits (one per issue):

  • loud pin-file loader (ggml/src/hydra-pins.h, header-only, shared by every consult path) + CUDA-side arming, one-line load summary, engagement counter, attach-time VRAM fit check with required/free/shortfall arithmetic
  • per-(layer, site) timing rings (n/p50/p99/mean us) with gpu/cpu/combine branches never merged, launch attribution apart, CUDA-event method contract
  • lookup-basis counters (hits/lookups, h) per (layer, site) + totals, decode-only labelled

Verification (certified base build-e0, stamp be4676f11 clean):

  • bridge B3 pool leg on this tree (disarmed): 27.4374 tok/s vs 27.3789 anchor (+0.21%), base CERTIFIED
  • armed boot + serve-probe: load summary exact (48 layers, 384 pins), engaged=0/armed=1, engagement gate CLOSED (exit 1) as designed pre-E1
  • bad pin path and bad pin line both abort loud (GGML_ABORT with reason); gate fixture self-test PASS (open=0 closed=1 unwired=2)
  • full llama-server build, zero nvcc warnings

No effect numbers in this PR (gate closed until E1 opens it).

Implements #142 behaviour fresh on the epic base: header-only
pin-file loader (ggml/src/hydra-pins.h, one format, one parser for
every consult path) with hard fail on fopen error and on any
unparseable line, load-time dedupe with counts, one-line load
summary (file, layers, total pins, per-layer min/max, dups
removed). Engagement counter exposed, incremented only via the
E1 hook. Attach-time VRAM fit check fails loudly with required,
free, and shortfall arithmetic.

Dedupe substance of #146 absorbed here (Refs #146); range-check
moves to the E1 validate hook hydra_pins_validate_range.

Assisted-by: Muse Spark
Implements #143 behaviour: per-(layer, site) rings (last 1024)
reporting n, p50, p99, mean microseconds per branch, with
gpu, cpu, and combine branches never merged. Kernel-launch
attribution kept apart from event-measured time. GPU branch
values must come from lazily harvested CUDA events (host timers
measure launch, not execution); CPU branch uses host wall time.
Instrument cost documented in the block contract.

Assisted-by: Muse Spark
Implements #144 behaviour: hits over lookups on the k-per-token
lookup basis, recorded per (layer, site) with h = hits/lookups
plus totals. Exactly one path records by construction on this
base, so no blended semantics are possible. Every dump line
labelled decode-only; callers documented as decode-gated.

Assisted-by: Muse Spark
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant