Consolidate the ported experiment code onto shared helpers (0.2.1)

CITATION.cff

@@ -23,7 +23,7 @@ authors:
     family-names: Lane
     affiliation: University of Cambridge
 license: MIT
-version: "0.2.0"
+version: "0.2.1"
 date-released: "2026-08-31"
 repository-code: "https://github.com/hematteo/sparse-readout-prism"
 preferred-citation:

README.md

@@ -66,6 +66,15 @@ script and command line in [`docs/REPRODUCE.md`](docs/REPRODUCE.md) §2):
   with the seed-variation and low-k training configs;
 - the sense-labelled evaluation on CoarseWSD-20.
 
+0.2.1 consolidates that code onto the repo's shared helpers (`research/cross_lens.py`,
+`research/wsd.py`, `research/row_geometry.py`; the library's checkpoint reader,
+top-k kernel and centering helper), fixes the small bugs found in review, records
+provenance in every result file, and exposes the choices the paper runs made
+implicitly as flags whose defaults reproduce the paper: `--centering {live,trained}`
+(full-vocabulary vs. training row mean), `--agreement-rule {half,strict}` and
+`--null-population {all,cross}` (cross-lens aggregators), and
+`--knn-exclude-self` (leave-one-out core recovery).
+
 ## Install
 
 Requires Python 3.11 or 3.12.
@@ -211,7 +220,8 @@ src/sparse_readout_prism/         core library
   research/                       flat helpers shared by several scripts/ entry
                                   points (Qwen readout + query-decomposition
                                   toolkits, prompt banks, registry, run IO,
-                                  seed-stability contrast pipeline);
+                                  seed-stability pipeline, row-geometry helpers,
+                                  cross-lens toolkit, CoarseWSD-20 helpers);
                                   logic used by a single script stays inline in
                                   that script
 
@@ -241,7 +251,6 @@ configs/
 data/
   query_banks/                    curated prompt banks (paper inputs)
   cross_lens/                     EN-ZH / EN-DE cross-lens prompt banks + exclusion report
-  wsd/                            optional sense-gloss anchors for the CoarseWSD-20 runs
   appendix/                       audited literals behind the Appendix K tables
   audit/                          feature-label audit annotations + counts (Appendix L)
 

configs/sweeps/README.md

@@ -38,7 +38,7 @@ Two kinds of file live here:
 | `deepseek_r1_distill_llama_8b_paper.yaml`, `…_16x_k128.yaml` | run config | R1-Distill-Llama-8B finalists. |
 | `ministral3_8b_paper.yaml`, `…_16x_k128.yaml` | run config | Ministral-3-8B-Base finalists (32×/k256 and strict-budget 16×/k128). |
 | `qwen35_2b_seedvar_base.yaml` | run config | Qwen3.5-2B seed-variation base (32×/k256, seed 0). The stability appendix's three-seed families and the multi-seed rows of the low-k table are this file with `--set` overrides of `run.seed` / `run.init_seed` / `data.data_seed`, `factorizer.d_features` / `factorizer.k` / `evaluation.k`, and `run.name` / `run.output_dir` (cells listed in the file header). |
-| `qwen35_0p8b_paper_topk_32x_k256_s1.yaml`, `…_s2.yaml` | run config | Qwen3.5-0.8B 32×/k256 seed-window cells (seeds 1 and 2; seed 0 is the archived finalist). |
+| `qwen35_0p8b_paper_topk_32x_k256_s1.yaml`, `…_s2.yaml` | run config | Qwen3.5-0.8B 32×/k256 seed-window cells (seeds 1 and 2; seed 0 is the archived finalist, `qwen0p8b_k256` in the exp2 registry / `qwen0p8b_k256_32x` in the fidelity registry). Recipe matched on the keys this trainer reads (see the file headers). |
 | `lowk_qwen35_2b_16x_base.yaml`, `lowk_qwen35_0p8b_16x_base.yaml` | run config | Low activation-budget cells at 16× width (`k` in {32, 64} via `--set factorizer.k=… --set evaluation.k=…`), behind the sparsity-budget table. |
 
 The provisional Gemma operating points in the registries were trained with the
@@ -56,10 +56,16 @@ by the run-config guard, before any data is loaded.)
 The run files behind the paper's seed-variation and low-k cells carried a few
 extra keys (`data.row_preprocessing`, `training.init_mode`, `lr_schedule`,
 `lambda_mode`, `final_lr_frac`, `auxk_*`, `artifacts.make_figures`) that no
-code path in the shipped trainer reads, and did not read in the trainer that
-produced those cells either; they are omitted here rather than shipped as if
-they were live. Every value the trainer does read is identical to the paper
-files.
+code path in the shipped trainer reads. The Qwen3.5-2B seed-variation and
+low-k cells were trained with this trainer, so for them the keys were inert;
+the two Qwen3.5-0.8B seed cells were launched from an earlier trainer tree
+that is not available to check, and the released paper dictionaries record
+the same keys in their checkpoint configs (see the `config` block of any
+Hub checkpoint), so treat "identical recipe" for those as covering the keys
+this trainer reads. The keys are omitted here rather than shipped as if they
+were live. The five seed-variation / low-k run configs carry
+`training.log_every: 250` and `training.checkpoint_every: 5000` (logging and
+checkpoint cadence only) for parity with `paper_phase2_base.yaml`.
 
 The cross-model finalist configs (R1-Distill-Qwen-7B, R1-Distill-Llama-8B,
 Ministral) are launched with the generic trainer —

configs/sweeps/lowk_qwen35_0p8b_16x_base.yaml

@@ -23,6 +23,9 @@
 # data.path is the {W_U_orig, h_LN} extraction of Qwen3.5-0.8B
 # (scripts/data/extract_model_readout.py), the artifact
 # configs/registries/result1_query_fidelity_cluster.yaml resolves for this model.
+# training.log_every / training.checkpoint_every (250 / 5000) are logging and
+# checkpoint cadence only, added for parity with paper_phase2_base.yaml; they do
+# not change the trained dictionary.
 
 run:
   name: lowk_qwen35_0p8b_16x
@@ -54,6 +57,8 @@ training:
   prism_top_n: 1
   lambda_prism: 0.001
   row_sampling: hybrid_50freq_50uniform
+  log_every: 250          # cadence only: parity with paper_phase2_base.yaml
+  checkpoint_every: 5000  # cadence only: parity with paper_phase2_base.yaml
 
 evaluation:
   k: 64

configs/sweeps/lowk_qwen35_2b_16x_base.yaml

@@ -26,6 +26,9 @@
 # data.path is the {W_U_orig, h_LN} extraction of Qwen3.5-2B
 # (scripts/data/extract_model_readout.py), the artifact
 # configs/registries/result1_query_fidelity_cluster.yaml resolves for this model.
+# training.log_every / training.checkpoint_every (250 / 5000) are logging and
+# checkpoint cadence only, added for parity with paper_phase2_base.yaml; they do
+# not change the trained dictionary.
 
 run:
   name: lowk_qwen35_2b_16x
@@ -57,6 +60,8 @@ training:
   prism_top_n: 1
   lambda_prism: 0.001
   row_sampling: hybrid_50freq_50uniform
+  log_every: 250          # cadence only: parity with paper_phase2_base.yaml
+  checkpoint_every: 5000  # cadence only: parity with paper_phase2_base.yaml
 
 evaluation:
   k: 64

configs/sweeps/qwen35_0p8b_paper_topk_32x_k256_s1.yaml

@@ -1,8 +1,17 @@
 # Qwen3.5-0.8B readout SAE, 32x/k=256 recipe, seed 1 (tab:app-k-qwen08b-seed-window).
-# Repeats the archived seed-0 dictionary setting (qwen0p8b_k256_32x in
-# configs/registries/exp2_selected_sae_checkpoints.yaml) with only
-# run.seed / run.init_seed / data.data_seed changed. Sister file:
-# qwen35_0p8b_paper_topk_32x_k256_s2.yaml (seed 2).
+# Matches the archived seed-0 dictionary setting (qwen0p8b_k256 in
+# configs/registries/exp2_selected_sae_checkpoints.yaml; the same checkpoint is
+# qwen0p8b_k256_32x in configs/registries/result1_query_fidelity_cluster.yaml)
+# on every key this trainer reads, with only run.seed / run.init_seed /
+# data.data_seed changed. Sister file: qwen35_0p8b_paper_topk_32x_k256_s2.yaml
+# (seed 2).
+#
+# The archived run file also carried keys this trainer does not read
+# (data.row_preprocessing, training.init_mode / lr_schedule / lambda_mode /
+# final_lr_frac / auxk_*, artifacts.make_figures). The seed-0 cell was launched
+# from an earlier trainer tree that is not available to check, so whether those
+# keys were live there is not verified; they are omitted here, and "same recipe"
+# means the keys this trainer reads.
 #
 # Launched as-is, no --set overrides:
 #   uv run python scripts/train/train_readout_sae_from_config.py \
@@ -11,6 +20,9 @@
 # d_features 32768 = 32 x d_model (1024). data.path is the {W_U_orig, h_LN}
 # extraction of Qwen3.5-0.8B (scripts/data/extract_model_readout.py), the artifact
 # configs/registries/result1_query_fidelity_cluster.yaml resolves for this model.
+# training.log_every / training.checkpoint_every (250 / 5000) are logging and
+# checkpoint cadence only, added for parity with paper_phase2_base.yaml; they do
+# not change the trained dictionary.
 
 run:
   name: qwen35_0p8b_paper_topk_32x_k256_s1
@@ -42,6 +54,8 @@ training:
   prism_top_n: 1
   lambda_prism: 0.001
   row_sampling: hybrid_50freq_50uniform
+  log_every: 250          # cadence only: parity with paper_phase2_base.yaml
+  checkpoint_every: 5000  # cadence only: parity with paper_phase2_base.yaml
 
 evaluation:
   k: 256

configs/sweeps/qwen35_0p8b_paper_topk_32x_k256_s2.yaml

@@ -1,8 +1,17 @@
 # Qwen3.5-0.8B readout SAE, 32x/k=256 recipe, seed 2 (tab:app-k-qwen08b-seed-window).
 # Sister file of qwen35_0p8b_paper_topk_32x_k256_s1.yaml: the archived seed-0
-# dictionary setting (qwen0p8b_k256_32x in
-# configs/registries/exp2_selected_sae_checkpoints.yaml) with only
-# run.seed / run.init_seed / data.data_seed changed.
+# dictionary setting (qwen0p8b_k256 in
+# configs/registries/exp2_selected_sae_checkpoints.yaml; the same checkpoint is
+# qwen0p8b_k256_32x in configs/registries/result1_query_fidelity_cluster.yaml)
+# on every key this trainer reads, with only run.seed / run.init_seed /
+# data.data_seed changed.
+#
+# The archived run file also carried keys this trainer does not read
+# (data.row_preprocessing, training.init_mode / lr_schedule / lambda_mode /
+# final_lr_frac / auxk_*, artifacts.make_figures). The seed-0 cell was launched
+# from an earlier trainer tree that is not available to check, so whether those
+# keys were live there is not verified; they are omitted here, and "same recipe"
+# means the keys this trainer reads.
 #
 # Launched as-is, no --set overrides:
 #   uv run python scripts/train/train_readout_sae_from_config.py \
@@ -11,6 +20,9 @@
 # d_features 32768 = 32 x d_model (1024). data.path is the {W_U_orig, h_LN}
 # extraction of Qwen3.5-0.8B (scripts/data/extract_model_readout.py), the artifact
 # configs/registries/result1_query_fidelity_cluster.yaml resolves for this model.
+# training.log_every / training.checkpoint_every (250 / 5000) are logging and
+# checkpoint cadence only, added for parity with paper_phase2_base.yaml; they do
+# not change the trained dictionary.
 
 run:
   name: qwen35_0p8b_paper_topk_32x_k256_s2
@@ -42,6 +54,8 @@ training:
   prism_top_n: 1
   lambda_prism: 0.001
   row_sampling: hybrid_50freq_50uniform
+  log_every: 250          # cadence only: parity with paper_phase2_base.yaml
+  checkpoint_every: 5000  # cadence only: parity with paper_phase2_base.yaml
 
 evaluation:
   k: 256

configs/sweeps/qwen35_2b_seedvar_base.yaml

@@ -28,6 +28,9 @@
 # d_features 65536 = 32 x d_model (2048). data.path is the {W_U_orig, h_LN}
 # extraction of Qwen3.5-2B (scripts/data/extract_model_readout.py), the artifact
 # configs/registries/result1_query_fidelity_cluster.yaml resolves for this model.
+# training.log_every / training.checkpoint_every (250 / 5000) are logging and
+# checkpoint cadence only, added for parity with paper_phase2_base.yaml; they do
+# not change the trained dictionary.
 
 run:
   name: qwen35_2b_seedvar_base
@@ -59,6 +62,8 @@ training:
   prism_top_n: 1
   lambda_prism: 0.001
   row_sampling: hybrid_50freq_50uniform
+  log_every: 250          # cadence only: parity with paper_phase2_base.yaml
+  checkpoint_every: 5000  # cadence only: parity with paper_phase2_base.yaml
 
 evaluation:
   k: 256

data/cross_lens/README.md

@@ -18,10 +18,10 @@ Every record has these keys (`lang_a` / `lang_b` only in the EN-DE bank):
 | `concept` | str | The concept the prompt elicits; used to exclude same-concept pairs from the shuffled-pairing null. |
 | `prompt` | str | The full prompt fed to the model; the final position is scored. |
 | `form_a` | str | The in-context answer form (the language or script the prompt asks for). |
-| `form_b` | str | The same concept's canonical form in the other language. |
-| `targets` | list[str] | Every surface the runner decomposes: `form_a`, `form_b`, any extra forms, then the nulls. |
-| `null_targets` | list[str] | Unrelated tokens (script- or language-matched) for the unrelated-token null floor. |
-| `answer` | str | The expected answer (`form_a` without its leading space). |
+| `form_b` | str | The same concept's canonical form in the other language. Controls in the EN-DE bank repeat `form_a` here (`7`, ` Paris`), since their surface does not depend on language. |
+| `targets` | list[str] | Every surface the runner decomposes, `null_targets` last. Opens with `form_a`; `form_b` follows, but extra forms may sit between them (`cle_03`: ` large`, ` big`, `大`; the three-lens bank lists the answer's own variants before the other-language forms) or after `form_b` (EN-ZH controls: ` 7`, `7`). EN-DE controls therefore list `form_a` twice (`['7', '7', ' Schuh', ' shoe']`). |
+| `null_targets` | list[str] | Unrelated tokens (script- or language-matched) for the unrelated-token null floor. Empty for the EN-ZH controls. |
+| `answer` | str | The expected continuation. Usually `form_a` without its leading space; for three EN-ZH items the answer is a multi-character word whose single-token first character is `form_a` (`clz_09` 睡觉 / 睡, `clz_16` 月亮 / 月, `exm_03` 红色 / 红), so the probed token is the answer's prefix. |
 | `lang_a`, `lang_b` | str | EN-DE bank only: the language of `form_a` / `form_b` (`de` or `en`). |
 
 Each target string is probed by the id of its first token, and the aggregators

docs/DATA.md

@@ -15,14 +15,12 @@ A record is one **case** — a prompt, a position to score, two targets
 (`A`, `B`), and an expected side. Every script that consumes a bank reads
 the same schema; the banks are model-agnostic.
 
-Two further paper-input sets live beside them. [`data/cross_lens/`](../data/cross_lens/)
+One further paper-input set lives beside them. [`data/cross_lens/`](../data/cross_lens/)
 holds the EN–ZH and EN–DE prompt banks of the cross-lens study (one record per
 prompt: two surface forms, an unrelated null target, family and control tags)
 together with the EN–DE cognate-exclusion report and the one-prompt bank of the
 three-lens worked example; schema and provenance are in
-[`data/cross_lens/README.md`](../data/cross_lens/README.md). [`data/wsd/`](../data/wsd/)
-holds the optional sense-gloss anchors that `scripts/run/run_wsd_feature_alignment.py`
-accepts through `--anchors`; the paper's CoarseWSD-20 runs did not use them.
+[`data/cross_lens/README.md`](../data/cross_lens/README.md).
 
 ## 2. Extracted readouts (per-model artifacts)