Skip to content

paper: pass-2 consistency — purge version labels from App A/B/bib#44

Merged
FluffyAIcode merged 1 commit intomainfrom
AgentMemory/paper-protocol-consistency-pass2-c478
Apr 24, 2026
Merged

paper: pass-2 consistency — purge version labels from App A/B/bib#44
FluffyAIcode merged 1 commit intomainfrom
AgentMemory/paper-protocol-consistency-pass2-c478

Conversation

@FluffyAIcode
Copy link
Copy Markdown
Owner

Reviewer feedback addressed

"论文的'评测协议'和'版本边界'在全文内部自相矛盾,影响可信度"

  • 摘要和正文把这篇稿子写成一个统一的 KakeyaLattice family paper,同时包含 D4 和 E8 两个变体
  • 正文主结果 §5 是 D4 + snapshot-mode + 4 passages;§6 的 E8 又切换成 in-forward + no-boundary + n=32 + 95% CI
  • §7 caveat 明确写了:"All numbers in this paper are snapshot-mode" — 与 §6 直接冲突
  • 附录的复现实验混入了 v1.5 rigorous evaluation harness;附录 B/C 又把 D4 写成 v1.4、E8 写成 v1.5

What was already fixed in prior PRs

The protocol self-contradiction in §7 caveat and the mixed protocol framing across §5/§6 were already fixed in PR #41 (paper: fix protocol self-contradiction and bit-accounting inconsistencies, merged to main as 9493fac). At main HEAD on 2026-04-24:

  • §1 has a "Strict-GPU, live-vLLM; two measurement protocols" paragraph declaring both protocols up-front.
  • §4 has a new §4.4 "In-forward rigorous evaluation" subsection defining n=32 / 95% CI.
  • §5 title: Main results: D_4 variant, snapshot protocol. §5 intro: All tables in this section use the snapshot protocol defined in §4.2 (n=4 passages, boundary-layer protection k=2). The protocol isolates codec intrinsic error...
  • §6 preamble: Every result in this section uses the in-forward rigorous evaluation protocol defined in §4.4.
  • §7.2 caveat no longer says "all numbers are snapshot-mode"; instead maps each protocol to its question: Snapshot-mode isolates codec intrinsic error and is used for every D_4 result in §5... In-forward rigorous measures deployment-realistic error and is used for every result in §6.
  • All results-table captions cite their protocol tag.

What this PR (Pass 2) fixes — the residual version-label leakage

Reviewer correctly noted that even after PR #41, the appendices and bibliography still leaked version labels and contradicted the paper's own framing choice (lattice-name-only in body). Specifically:

App A (Reproducibility manifest)

  • Opening line said introduced with the v1.5 release, framing the harness as a v1.5-era artifact instead of a codec-agnostic, protocol-configurable tool.
  • Code-block comment said # PPL / MSE / CR (v1.4 + v1.5 + TurboQuant at matched Q / b), treating v1.4 / v1.5 as codec identities.
  • Directory paths reports/v1_5_release/ / reports/v1_4_release/ referenced without explaining that these are on-disk git-tag directory names.

Fixed: rewrote opening paragraph (both lattice variants), code-block comment (# In-forward rigorous (n=32, 95% CI): D4, E8, and TurboQuant, same run), --out-dir defaults (now reports/rigorous_eval/, protocol-generic). Added closing paragraph explaining that --v15-q-values is a CLI-flag identifier and reports/v1_{4,5}_release/ are git-tag-matched directory paths, not codec names.

App B (was "Canonical naming", now "Implementation identifiers")

Fixed: Renamed to "Implementation identifiers". First sentence: The paper is agnostic to version labelling: the two codec variants are the D_4 nested lattice and the E_8 nested lattice, and every result table cites its protocol. This appendix lists only the repository-level identifiers needed to reproduce the bit-identical codec output. Removed all "spoken / written" entries. Final disclaimer: The v1.4 / v1.5 strings appear only as git release tags and as the V14/V15 class-name prefixes chosen by the repository authors; the paper itself references the codec variants exclusively by their lattice names.

Bibliography (kakeya-v14-release, kakeya-v13-paper)

  • KakeyaLattice v1.4: the canonical implementationKakeyaLattice: the canonical implementation (drop version suffix in title).
  • Package reference kakeyaturbo_py.V14KakeyaZamirLatticeGPUPython package kakeyalattice (classes V14KakeyaZamirLatticeGPU and V15KakeyaZamirE8GPU). kakeyaturbo_py was renamed to kakeyalattice in PR paper: remove version labels from title/body, present as D4/E8 lattice variants #40; this was a regression.
  • Harness path benchmarks/multimodel_v14_vs_tq.pybenchmarks/rigorous_eval.py (the actual current canonical harness).
  • Snapshot-hook path vllm_backend/kakeya_v1_3_ppl/snapshot_hook.pyvllm_backend/kakeya_v1_4_snapshot/snapshot_hook.py (matches the post-rename repository layout).
  • Superseded by this paper at tag v1.4.Prior unpublished draft, superseded by this paper. (removed version-label leakage from the prior-work reference).

What remains literal v1.4 / v1.5 — intentional, disclaimed

  • Bib entry kakeya-v14-release: lists Release tags: v1.4 (first D_4 release, commit 6b02711) and v1.5 (first E_8 release). These ARE the tags the bib points at.
  • App A closing paragraph: explicitly labels --v15-q-values and reports/v1_{4,5}_release/ as repository-level identifiers, disclaiming their role as codec identities.
  • App B: class names V14KakeyaZamirLatticeGPU / V15KakeyaZamirE8GPU; module names kakeyalattice.v1_4_kakeya_zamir_lattice_gpu / kakeyalattice.v1_5_kakeya_zamir_e8_gpu; release tags. These are the real Python symbols and git refs a reader needs to reproduce bit-identical codec output. The closing disclaimer in App B explicitly notes that these are repository artifacts.

Post-fix audit

$ rg -n "v1\.4|v1\.5" reports/paper/kakeyalattice.tex
1358:  Release tags: \texttt{v1.4} (first $D_4$ release, commit
1359:  \texttt{6b02711}) and \texttt{v1.5} (first $E_8$ release).
1519:$E_8$-variant codec registered at release tag \texttt{v1.5};
1539:    release tag \texttt{v1.4} (commit \texttt{6b02711}).
1543:    release tag \texttt{v1.5}.
1546:The \texttt{v1.4} / \texttt{v1.5} strings appear only as git release

Every remaining v1.4 / v1.5 is either a bib entry release tag, a class/module reproducibility identifier, or the explicit disclaimer that the paper uses lattice names only. Zero body-prose or table-cell leakage.

Build

$ cd reports/paper
$ rm -f *.aux *.log *.out *.toc *.fls *.fdb_latexmk *.pdf
$ latexmk -pdf -interaction=nonstopmode -halt-on-error kakeyalattice.tex
...
Output written on kakeyalattice.pdf (15 pages, 452079 bytes).

15 pages (unchanged from PR #42 trim), 452 KB (+1 KB from new App A explanatory paragraph), zero undefined references, zero LaTeX errors.

Files

  • reports/paper/kakeyalattice.tex (+55 −39 lines)
  • reports/paper/kakeyalattice.pdf (regenerated)

No code / hook / codec / benchmark / release-artifact changes in this PR.

Not for auto-merge

Per user's "先不要合入" pattern from the Stage 0.5 PR #43 thread, this PR is filed as draft. Review the App A paragraph + App B structure; merge when satisfied with the reproducibility-vs-brand-consistency trade-off.

…A/B/bib

Second-pass cleanup after reviewer flagged residual v1.4 / v1.5 string
leakage in reproducibility / naming appendices and bibliography.

FINDINGS AT START OF THIS PASS

After PR #40 (lattice-name framing) and PR #41 (protocol-consistency),
the body prose and table captions were fully clean. But reviewer
correctly noted that:

  - App A (Reproducibility manifest) prose said 'introduced with the
    v1.5 release', framing the canonical harness as a v1.5-era
    artifact rather than a codec-agnostic harness.
  - App A code-block comment said '(v1.4 + v1.5 + TurboQuant at
    matched Q / b)', treating v1.4 / v1.5 as codec identities.
  - App A directory paths reports/v1_5_release/ and
    reports/v1_4_release/ were referenced without explaining that
    these are on-disk git-tag directory names, not codec labels.
  - App B (called 'Canonical naming') used v1.4 / v1.5 as the
    *canonical name* of each codec, with spoken-form entries
    ('v1.4 kakeya zamir lattice GPU') that contradicted the paper's
    own framing choice (the paper references lattice variants
    exclusively by D_4 / E_8 names).
  - Bibliography entry kakeya-v14-release:
    - referenced old package name kakeyaturbo_py (renamed to
      kakeyalattice in PR #40)
    - pointed at the old snapshot-hook path
      vllm_backend/kakeya_v1_3_ppl/snapshot_hook.py (renamed to
      kakeya_v1_4_snapshot in an earlier cleanup)
    - used 'Superseded by this paper at tag v1.4' for the prior-work
      reference, which is both inaccurate now and leaks version-label
      semantics.

FIXES

App A (Reproducibility manifest):
  - Open line rewritten: 'The full multi-model benchmark for both
    lattice variants uses the in-forward rigorous evaluation harness
    (§4). The snapshot-protocol D_4 tables in §5 are reproducible by
    the same harness with --mode snapshot --boundary-size 2
    --n-passages 4.' Frames the harness as codec-agnostic and
    protocol-configurable, not as a v1.5-specific artifact.
  - Code-block comment rewritten: '# In-forward rigorous (n=32, 95%
    CI): D4, E8, and TurboQuant, same run. --q-values selects D4
    operating points; --v15-q-values selects E8 operating points;
    --tq-b-values selects TurboQuant bit widths.' Explicit that CLI
    flag names are repository identifiers, not codec names.
  - --out-dir changed from reports/v1_5_release to
    reports/rigorous_eval (protocol-generic).
  - Closing paragraph explains that --v15-q-values and
    reports/v1_4_release/ / reports/v1_5_release/ strings are
    repository-level identifiers (CLI flags, on-disk paths matching
    git release tags) listed here solely for bit-level reproducibility.

App B (renamed 'Canonical naming' -> 'Implementation identifiers'):
  - Replaced the sub-bullet-heavy 'v1.4 codec (D_4)' / 'v1.5 codec
    (E_8)' structure with a flat list where each codec variant is
    named by lattice only, and the class / module / release-tag
    strings are listed as implementation details (one line each, not
    as 'spoken / written' canonical names).
  - Removed 'spoken / written: v1.4 kakeya zamir lattice GPU' and
    'v1.5 kakeya zamir E8 GPU' entries entirely — these were the
    only remaining prose suggesting version labels are canonical.
  - Added explicit disclaimer: 'The v1.4 / v1.5 strings appear only
    as git release tags and as the V14/V15 class-name prefixes
    chosen by the repository authors; the paper itself references
    the codec variants exclusively by their lattice names.'

Bibliography (kakeya-v14-release, kakeya-v13-paper):
  - Title: 'KakeyaLattice v1.4: the canonical implementation' ->
    'KakeyaLattice: the canonical implementation' (drop the v1.4
    version suffix).
  - Package reference: kakeyaturbo_py.V14KakeyaZamirLatticeGPU ->
    'Python package kakeyalattice (classes
    V14KakeyaZamirLatticeGPU and V15KakeyaZamirE8GPU)'.
  - Harness path: benchmarks/multimodel_v14_vs_tq.py ->
    benchmarks/rigorous_eval.py (the actual current harness).
  - Snapshot-hook path: vllm_backend/kakeya_v1_3_ppl/snapshot_hook.py
    -> vllm_backend/kakeya_v1_4_snapshot/snapshot_hook.py (matches
    the post-rename repository layout).
  - Release tags: kept as bib metadata 'v1.4 (first D_4 release,
    commit 6b02711) and v1.5 (first E_8 release)' — these remain
    git-tag metadata appropriate for a bib entry.
  - kakeya-v13-paper: 'Superseded by this paper at tag v1.4.' ->
    'Prior unpublished draft, superseded by this paper.' (removes
    version-label leakage from the prior-work reference).

WHAT REMAINS LITERAL v1.4 / v1.5 (intentional)

  - bib entries: kakeya-v14-release lists git release tags v1.4 and
    v1.5 — these ARE the tags the bib points at.
  - App A closing explanatory paragraph: explicitly labels
    --v15-q-values and reports/v1_{4,5}_release/ as repository-level
    identifiers, disclaiming their role as codec identities.
  - App B: class names V14KakeyaZamirLatticeGPU /
    V15KakeyaZamirE8GPU; module names
    kakeyalattice.v1_4_kakeya_zamir_lattice_gpu /
    kakeyalattice.v1_5_kakeya_zamir_e8_gpu; release tags v1.4 / v1.5.
    These are the real Python symbols and git refs a reader needs to
    reproduce bit-identical codec output; removing them would break
    reproducibility. The closing disclaimer in App B explicitly
    notes that these are repository artifacts, not codec names.

BUILD

latexmk -pdf clean rebuild: 15 pages (unchanged from PR #42 trim), 452
KB (+1 KB due to new App A explanatory paragraph), zero undefined
references, zero LaTeX errors. 94 lines changed in .tex.

CROSS-REFERENCES ALREADY FIXED IN PRIOR PRS (kept for reviewer
confidence):

  - §7 caveat 'All numbers in this paper are snapshot-mode' was
    removed in PR #41.
  - §1 Strict-GPU paragraph and §1 paper-structure paragraph
    explicitly cover both protocols (PR #41).
  - §4 has new §4.4 'In-forward rigorous evaluation' defining the
    n=32 / 95% CI protocol (PR #41).
  - §5 title is 'Main results: D_4 variant, snapshot protocol';
    §5 intro paragraph says every table uses snapshot (PR #41).
  - §6 preamble says 'Every result in this section uses the
    in-forward rigorous evaluation protocol defined in §4.4'
    (PR #41).
  - All result-table captions cite their protocol (PR #41).

This PR is DRAFT; DO NOT merge automatically per user instruction.
@FluffyAIcode FluffyAIcode marked this pull request as ready for review April 24, 2026 10:57
@FluffyAIcode FluffyAIcode merged commit 4787334 into main Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants