v3.46 · session-layer viability: prefix channel (A/C) is the ship track; B_ams_text is a diagnostic

[FluffyAIcode]

Child of #28. Decision-framework spike per SPRINT_CLOSEOUT_v3.46.md §10.

Status: complete. Fresh-init (CPU + GPU), trained-ckpt (GPU), framing correction, and the corrected final decision are all in this PR.

---

Framing (read this first)

AMS is not a RAG system and is not a knowledge graph. Its core mechanism is a prefix / hidden-state injection channel: memories are encoded into continuous vectors delivered into the backbone's forward pass without going through text re-prompting. That channel is what A_ams_prefix and C_ams_hybrid exercise in this benchmark.

• RAG = retrieve text, prepend to prompt. B_flat_cos and B_ams_text are RAG-shaped modes in this benchmark. They are retained as diagnostics and upper bounds, not as product candidates.
• Knowledge graph = explicit entities / relations / symbolic query. AMS has none of these; DirectionTree is a continuous-embedding routing structure.

An earlier revision of this PR concluded "ship B_ams_text as cheaper RAG backend, move P0–P4 to research track." That was a framing error. It is retracted in SPRINT_CLOSEOUT_v3.46.md §10.9 and replaced with the decision below.

Protocol

• session_viability.py (~550 lines): 5-mode benchmark on a synthetic session.
• 10 targeted-recall queries over a growing store of facts. Hit = expected_keyword substring in answer.
• Backbone: Qwen2.5-1.5B-Instruct (bf16). Two store sizes: N=10 (identity-only) and N=20 (+10 distractors), same 10 queries.
• No Cfg changes, no loss changes, no SUT changes. All existing APIs: MemLLM.write, prepare_decode_context, MemLLM.generate, layer_pool, _compute_content_semantic_emb.
• Honors AMS_TRAINED_WEIGHTS via the loader added in v3.46-trained: 60-step training + audit (18/26, −3 vs fresh 21/26) #28.
• Trained runs on NVIDIA H200 (vast.ai). train_v346.py --steps 60 → ckpt/v346_trained.pt in 193.5 s. post_probe = {tail_head_slot1_abs_mean: 7.25e-4, vocab_proj_last_abs_mean: 5.09e-4} (handoff expected 7.30e-4 / 5.49e-4; matches within <10 % noise).
• Loader verification on both trained runs: loaded=202 skipped=0 shape_errs=0 provenance=AgentMemory/v346-revertE-topk-nonexclusive-7e97.

What each mode measures:

Mode	Category	What it tells us
D_full_history	no retrieval, full context	Ceiling. The backbone's own answer quality when it has all facts in the prompt.
B_flat_cos	RAG-shaped	Does a naïve cosine-over-semantic_emb retrieve the right fact?
B_ams_text	RAG-shaped	Does AMS's full retrieval stack (DirectionTree + gate + rerank) retrieve the right fact? Upper bound on the retrieval side.
A_ams_prefix	AMS core mechanism	Does AMS's prefix channel deliver the memory into the forward pass without re-prompting?
C_ams_hybrid	AMS core + 1 crutch text memory	Same mechanism with a single text memory as scaffolding — an intermediate diagnostic.

The B modes are not product candidates. They are there to isolate whether a failure lives on the retrieval side or the prefix side.

Data

Fresh-init CPU (reports/session_viability_fresh/, ..._20facts/)

N=10

Mode	Hit-rate	avg in-tokens	avg retrieve ms	avg generate ms
D_full_history	100%	159	0.0	4138
B_flat_cos	80%	55	144	4187
B_ams_text	70%	56	526	4030
A_ams_prefix	60%	11	453	19722
C_ams_hybrid	70%	26	471	21147

N=20

Mode	Hit-rate	avg in-tokens	avg retrieve ms	avg generate ms
D_full_history	100%	301	0.0	4590
B_flat_cos	70%	55	119	3954
B_ams_text	90%	54	544	4025
A_ams_prefix	60%	11	473	18502
C_ams_hybrid	70%	26	455	20320

Fresh-init GPU (H200, reports/session_viability_fresh_gpu/, ..._20facts/)

N=10

Mode	Hit-rate	avg in-tokens	avg retrieve ms	avg generate ms
D_full_history	100%	159	0.0	516
B_flat_cos	80%	55	31	507
B_ams_text	80%	55	416	386
A_ams_prefix	50%	11	500	14898
C_ams_hybrid	70%	27	428	15363

N=20

Mode	Hit-rate	avg in-tokens	avg retrieve ms	avg generate ms
D_full_history	100%	301	0.0	511
B_flat_cos	70%	55	32	486
B_ams_text	80%	55	411	370
A_ams_prefix	50%	11	513	15096
C_ams_hybrid	70%	27	450	15311

Trained ckpt (H200, 60 steps, reports/session_viability_trained/, ..._20facts/)

N=10

Mode	Hit-rate	avg in-tokens	avg retrieve ms	avg generate ms	Δ fresh-GPU hit	Δ fresh-GPU gen-ms
D_full_history	100%	159	0.0	537	0	+21
B_flat_cos	80%	55	31	526	0	+20
B_ams_text	80%	55	415	376	0	−9
A_ams_prefix	50%	11	452	13033	0	−1865
C_ams_hybrid	70%	27	466	14520	0	−843

N=20

Mode	Hit-rate	avg in-tokens	avg retrieve ms	avg generate ms	Δ fresh-GPU hit	Δ fresh-GPU gen-ms
D_full_history	100%	301	0.0	539	0	+28
B_flat_cos	70%	55	32	498	0	+13
B_ams_text	80%	55	417	373	0	+3
A_ams_prefix	50%	11	435	12900	0	−2196
C_ams_hybrid	70%	27	447	13853	0	−1458

What the data says about AMS's core mechanism

Read under the corrected frame (§10.9 in SPRINT_CLOSEOUT_v3.46.md):

1. A_ams_prefix at 50 % N=20, trained or fresh, is the actual state of AMS's mechanism as of v3.46. 60 steps of the §5.3 rotating 6-sentence corpus moves vocab_proj weights by ~5e-4 abs mean and does not measurably change any hit-rate. That is a scale/data finding, not a mechanism failure — but the mechanism is visibly unfinished.
2. B_ams_text at 80–90 % N=20 is an upper-bound diagnostic on the retrieval side. It says: given the right source_text in the prompt, Qwen2.5-1.5B can answer 8–9 / 10 queries. That tells us DirectionTree + gate + rerank are finding the right memory; the remaining gap lives in the prefix channel, not in retrieval.
3. Prefix-channel deficit: B_ams_text − A_ams_prefix = 30 points at N=20. This is the gap P0–P4 has to close. It is the single number that defines "AMS works" vs "AMS is a nicer embedding RAG".
4. A/C generate time is ~25× text modes on H200, not the ~5× seen on CPU. CFG double-forward + logit-shaping dominates wall time on fast hardware. This is architectural, not a regression; it is part of what P0–P4 needs to address (or justify).

Corrected decision (supersedes the earlier "ship B_ams_text" wording)

1. P0–P4 blackbox-audit work is the ship track. It is the only track that exercises AMS's actual mechanism. The earlier "move P0–P4 to research track" wording was a framing error, retracted in §10.9.
2. B_ams_text is a retained diagnostic and upper bound, not a product line. It does not ship as an AMS artifact — doing so would be shipping a RAG product with AMS-flavored retrieval, which is not what AMS is.
3. Success bar restated correctly: AMS is done when A_ams_prefix or C_ams_hybrid matches D_full_history at a large fraction of D's token cost. Beating B_ams_text is a necessary intermediate milestone, not the finish line, because B_ams_text is a different product category.
4. Immediate next experiment (a follow-up PR, not this one): 10× more training on a real corpus, re-run A_ams_prefix / C_ams_hybrid at N=10 / N=20. This tests whether the 30-point prefix-channel deficit is a scale problem or a mechanism problem. The current 60-step / 6-sentence budget cannot distinguish the two.

Notes

• CPU vs GPU cross-cut: GPU fresh-init B_ams_text at N=20 is 80 %, not the 90 % observed on CPU in §10.3. Greedy decode is deterministic given identical tokens, but bf16-vs-fp32 numerics in layer_pool + _compute_content_semantic_emb shift one borderline query (davis) across the retrieval gate. Not a bug in session_viability.py.
• A_ams_prefix hit-rate is 50 % on GPU vs 60 % on CPU at N=10 — same single-turn noise class on a 10-query session.

What this PR does NOT claim

• It does not claim AMS is ready to ship as a product today. A_ams_prefix = 50 % at N=20 is not a shippable number.
• It does not claim P0–P4 will work with more training; only that the current 60-step / 6-sentence training budget cannot move the prefix channel at all, so that budget cannot be used to conclude anything about the ceiling.
• It does not benchmark AMS against any RAG system. AMS is not RAG; the comparison is not apples-to-apples and is not done here.

Commits

• 153069c — scaffolding (5 modes, synthetic 20-turn session, JSON+MD report)
• dbbf850 — fix: write-gate drop + correct query embedding via layer_pool + content_sem_emb
• ed9e8e3 — N=10 fresh-init CPU results
• 6c2eec6 — N=20 distractor run
• 6692510 — SESSION_VIABILITY_HANDOFF.md for next GPU agent
• fd487f6 — GPU runs: trained ckpt + fresh-GPU baseline, N=10/20, 12 report files
• a4fc23d — §10.7/§10.8 tables and first-pass decision (now superseded by §10.9)
• 4e85eb3 — §10.9 framing correction: AMS is not RAG, not a KG; P0–P4 back on ship track; B_ams_text is a diagnostic, not a product