feat(vindex3): the true logit lens through the executor's own head (V3-LENS-1) - #487
Merged
Merged
Conversation
chrishayuk
force-pushed
the
v3-observe-cli
branch
from
September 20, 2026 14:41
6e34a2f to
4939706
Compare
…xecutor's own head
…3-LENS-1) One head path: PreparedOperands::head_logits (prepared final norm, then the prepared output head with its multiplier and softcap) and head_over_normed; the decode exit and all four batch exits call them, so a lens at the last layer is the executor's logits by construction. LogitLens is a consumer of the carrier tap that reads the layer output (layer scale applied) at declared sites, records each declared token's log-probability and rank under the full log-softmax, and counts its head passes. Readouts ride on the run record with their price on the receipt; `vindex3 observe` gains --lens-tokens/--lens-layers/--lens-attention/ --lens-top-k and prints stepping time and a per-depth table. Witnessed: parity with the lens armed everywhere on both CPU backends; the last-layer anchor bit-identical, including through the Gemma 4 miniature's layer scale; proper distributions at every depth. Gemma 3 4B, "The capital of France is": " Paris" rank 14055 at L0, 10 at L23, 1 at L24, near-certain by L26. Cost measured: 7.1x the forward on Granite, 30.8x on Gemma (one head pass costs a forward there), so sites must be declared. Freeze, amendments and results in docs/v3-lens-1-logit-lens.md.
…mary as lines The coverage gate found the LensReader impl unreached by larql-vindex's own tests (only the inference crate used it); the mutants job found the CLI's summary and lens table unassertable because they printed straight to stdout. The reader now has a trait-object witness, and the summary is built as lines a test reads before they are printed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
V3-LENS-1, rung 3 above V3-OBS-1 (#484) and V3-STREAM-1 (#485), stacked on the observe verb (#486; retargets to main once that merges).
PreparedOperands::head_logits(prepared final norm, then the prepared output head with multiplier and softcap) andhead_over_normed. The decode exit and all four batch exits now call them; the private batch helper is gone. A lens at the last layer is therefore the executor's logits by construction, not by agreement.LogitLensreads the layer output (layer_scale × after) at declared sites (all,every:<k>, a list; FFN and/or attention), records each declared token's log-probability and rank under a full f64 log-softmax plus the top-k, counts its head passes, and records its first failure instead of panicking in a callback.LensReaderlets the run record hold any reader.EventKind::Readout, sequenced between a write's stats and its structural event;head_passesandlens_failureon the receipt.vindex3 observe --lens-tokens … [--lens-layers …] [--lens-attention] [--lens-top-k …], plus stepping wall time and a per-depth table for the final position.Evidence (
docs/v3-lens-1-logit-lens.md, frozen before implementation)LP1 parity with the lens armed on every site, both CPU backends. LP2 anchor: the last FFN site's lens is bit-identical to the executor's logits on both backends and through the Gemma 4 miniature's layer scale. LP3: every decode/batch/observed parity gate bit-identical after the exit refactor. LP4: proper distributions at every depth. LP6: readouts replay equal; a failing lens is named on the receipt. LP7, Gemma 3 4B IT,
The capital of France is, final prompt position:Parisrank 14055 at L0 → 10 at L23 → 1 at L24 (logp −0.36) → near-certain by L26;Francepeaks around rank 400–900 in L16–20 and never leads; the exit row equals the executor's top-1. LP5 cost, single runs: Granite 183.5 → 1305.7 ms/position with all 40 FFN sites armed (7.1×), Gemma 134.0 → 4132.0 ms/position with 34 (30.8×); the forecast's "several times" was an underestimate and is recorded as such.Gates
fmt, clippy (larql-vindex, larql-inference, larql-cli), doc references, larql-vindex lib 4781, larql-inference lib 1544, larql-cli 917.