v3.48 stacked attention-sharing mechanisms 1+2+3+4: 120-step train + audit = 19/26 (prediction partially refuted)

[FluffyAIcode]

Scope

Training driver + runner measurement extension. No SUT code change.

What this PR does

1. New train_v348.py that activates four attention-sharing mechanisms simultaneously:
   • M1: Cfg(use_memory_context_encoder=False) + loss reweight (encoder_throughput 1.5→3.0, tail_semantic_anchor 0.5→0.1, etc.)
   • M2: Qwen layer-0 q_proj/k_proj/v_proj warm-start into QFormer layer-0 cross-attention (K/V tiled 6× from 256-dim GQA to 1536-dim MHA expectation)
   • M3: Per-step distillation loss pulling bridge.proj(f).mean(1) toward Qwen content-token hidden_mean; second optimizer over bridge.proj params only
   • M4: bridge.proj.q initialized from Qwen content-token hidden_mean of random corpus texts
2. v331_blackbox_eval.py 4.24 primary reader updated to follow SUT fallback chain (context_descriptor else semantic_emb)
3. 120-step CPU training → ckpt/v348_stacked.pt (not tracked)
4. 26-case audit under AMS_DETERMINISTIC=1

Training convergence (4 mechanisms delivered, measurable)

metric	v3.44-Trained (60 steps)	v3.48 (120 steps)	delta
total_loss	44.0	17.5	2.5× deeper
recon_loss	4.8	2.08	2.3× lower
vocab_anchor	−0.22	−0.33	50% deeper
bridge↔Qwen-pool cos (M3)	—	0.77 sustained, 0.87 peak	new signal

Audit result: 19/26 pass

transition	count	cases
FAIL → PASS	0	—
PASS → FAIL	0	—
Persistent PASS	19	(unchanged)
Persistent FAIL	7	4.7, 4.11, 4.13, 4.16, 4.19, 4.23, 4.24

Prediction made in v3.47 ("landing M1 → 20/26") is partially refuted: the mechanism-1 diagnostic still returns 0.812 LOO NN on 4 domains (matches prediction), but the 4.24 primary did not transition and M2+M3 regressed 4.23 (median rank 759 → 1089) and 4.24 heldout (0.875 → 0.750).

Mechanism diagnosis

mechanism	effect observed	net impact
M1 disable learned encoder	diagnostic metric 0.812/0.875 as predicted	would be +1 if primary reader fixed
M2 Qwen K/V warm-start	distill cos → 0.87 peak	−1 on 4.23 (tail slot direction pulled toward Qwen mean)
M3 distill target = hidden_mean	distill cos sustained 0.77	−1 on 4.24 heldout (target direction is domain-invariant)
M4 pool-init queries	bridge.proj.q L2 = 0.81	neutral

Root diagnosis

Qwen's content-token hidden_mean works as a final clustering signal (v3.47 diagnostic: 0.812 on 4.24) but fails as a distillation target for the prefix-generation pipeline: its principal components are dominated by "English declarative sentence" geometry, not topic geometry. Pulling bridge output toward it makes the system less domain-discriminative, hurting 4.23 (needs rare-keyword direction) and 4.24 heldout (needs domain separation).

Training signal was strong; destination was wrong.

Falsifiable v3.49 options (not in this PR)

1. Revert M2+M3, keep M1+M4: predicted 20/26
2. Change M3 target from hidden_mean to wte_centroid_of_strict_content_starters (domain-discriminative): predicted ≥ 20/26, possibly higher
3. Fix 4.24 primary reader to uniformly follow SUT fallback: predicted 20/26 on current ckpt

Artifacts

• train_v348.py (new file)
• ckpt/v348_stacked.pt (453 MB, not tracked; reproducible from python3 train_v348.py --steps 120)
• ckpt/v348_train_log.jsonl, ckpt/v348_train_stdout.log
• reports/v348_stacked_blackbox/{report.json, report.md, runner.log, audit_feedback.md}

Dependencies

Builds on PRs #18, #19, #20, #21. Clean fast-forward if they merge first.