Skip to content

docs(readme): bind the field-comparison table to primary sources (B4/L-1) - #291

Merged
Gavin-Borges merged 2 commits into
mainfrom
docs/readme-competitor-table-citations-b4-l1
Aug 26, 2026
Merged

docs(readme): bind the field-comparison table to primary sources (B4/L-1)#291
Gavin-Borges merged 2 commits into
mainfrom
docs/readme-competitor-table-citations-b4-l1

Conversation

@Gavin-Borges

Copy link
Copy Markdown
Owner

Closes B4 / L-1 - the README "SESTRAV vs Field" table's 36 competitor cells (PredIG / PRIME / NetMHCpan / pVACtools x 9 capability rows) carried no citation of any kind, and had not been checked against their subjects' own sources since the table was written. This is the largest reader-facing integrity gap on the board and a pre-tag blocker.

Every tool's own paper and GitHub README/docs were read directly. Seven cells corrected, twenty-nine confirmed accurate as already written, one deliberately left unresolved.

Corrections

Tool Row Was Now Source
PredIG Open source, pip-installable Open source (GPL-2.0), not pip-installable - Docker/Singularity or webserver PMC12613480 Code/Data availability; repo README
PredIG Pan-allele training Partial - "PredIG performs pan HLA-I allele predictions" PMC12613480 Methods
PRIME Open source, pip-installable Academic/non-commercial only; precompiled binary or build-from-source PRIME README, License + Installation
PRIME Antigen processing as training features Partial - its 28-node input layer carries no cleavage/TAP feature PMC9811684 STAR Methods
PRIME Pan-allele training Partial - an "expanded" but still finite allele list PRIME README v2.1 changelog
NetMHCpan End-to-end workflow Partial - auto-digests a submitted FASTA proteome, ranks by %Rank PMC7319546; DTU 1-Submission.php
pVACtools Antigen processing as training features Partial - NetChop/NetMHCstabPan are optional post-hoc annotation, not training input pvactools.readthedocs.io
pVACtools End-to-end workflow ✓ (neoantigens) same, plus qualifier - the authors' own Abstract says "when paired with a well-established genomics pipeline" PMC7056579 Abstract

Note the direction on two of these. The PredIG and PRIME open-source/pip corrections overstated a competitor's installability - the opposite of this project's usual error direction. .claude/rules/third-party-claims.md observes that all six prior instances leaned toward flattering SESTRAV; these two did not, which is worth recording rather than smoothing over.

The unresolved cell, and why it stays unresolved

NetMHCpan "Multi-virus support" (Pan-pathogen) is NOT corrected and NOT left quietly as-is. It is flagged in a README footnote and in docs/claims_register.md D34, pending your ruling.

Neither primary source discusses peptide biological origin (viral / bacterial / tumor / self) or uses the term "pan-pathogen" anywhere. The paper's only pan-specificity sentence - "given the pan-specific nature of both methods, predictions can be run for any MHC molecule of known sequence" - supports MHC-molecule breadth, which is a different claim.

Per rule 3, the claim does not get made when the source cannot support it. That cut both ways here: every synthesized replacement considered was itself an inference beyond what either source states, so the right claim could not be sourced either - which is precisely why this is a ruling for you rather than a rewrite by me.

Verification, and two process defects caught

Research ran as a fan-out (one agent per tool, all reading primary sources), then every finding was re-checked by an independent adversarial pass instructed to refute it. That second pass earned its place:

  • It caught two citations containing a fabricated or misattributed quotation attributed to NetMHCpan's own paper - a "length distribution of naturally presented peptides" clause and an "any peptide of known sequence" clause, neither of which appears in PMC7319546. This is the direct reason the Multi-virus cell above is unresolved rather than published on the first pass's citation.
  • Separately, I caught a transcription bug in the raw research output: one agent recorded pVACtools's "Pan-allele training" current-state as SESTRAV's own column text (No - ten fixed HLA-A/-B binding columns). That is an artifact, not a finding about pVACtools. That row was excluded from this pass entirely rather than corrected from a guess - pVACtools's pan-allele cell is unchanged and explicitly unverified by this pass, and D34 says so.

Both defects are recorded in D34 rather than quietly dropped, because they are exactly the class that file exists to catch.

Harness: the full integrity harness returns byte-identical 134 PASS / 0 WARN / 18 FAIL / 7 SKIP on this branch and on bare origin/main - measured both ways, with and without this diff, so the pre-existing 18 (which #288 addresses separately) are demonstrably not mine. Doc line-citation and commit-reference gates pass; git diff --check clean; zero banned typographic characters in any added line; full fast test suite green via the pre-push gate.

…L-1)

The 'SESTRAV vs Field' table's 36 competitor cells (PredIG/PRIME/NetMHCpan/
pVACtools x 9 rows) carried no citation of any kind - the largest
reader-facing integrity gap on the board and elevated to a pre-tag blocker.

Each tool's own paper and GitHub README/docs were read directly and every
cell checked against it. Seven cells were corrected:

- PredIG open-source/pip-installable: open source (GPL-2.0) but distributed
  only via Docker/Singularity or a webserver, no pip package.
- PredIG pan-allele training: Partial -> Yes, the paper states directly
  'PredIG performs pan HLA-I allele predictions.'
- PRIME open-source/pip-installable: academic/non-commercial license only,
  precompiled binary or build-from-source, no PyPI package.
- PRIME antigen processing as training features: Partial -> No, its 28-node
  input layer carries no proteasomal-cleavage or TAP feature.
- PRIME pan-allele training: Yes -> Partial, its own training/validation
  allele list is 'expanded' (still finite), a narrower claim than the
  generalize-to-unseen-alleles architecture the row means elsewhere.
- NetMHCpan end-to-end workflow: No -> Partial, it auto-digests a submitted
  FASTA proteome and ranks the resulting peptides by %Rank.
- pVACtools end-to-end workflow: added a qualifier - the authors' own
  Abstract requires pairing with an external variant-calling pipeline.
- pVACtools antigen processing as training features: Partial -> No, NetChop/
  NetMHCstabPan run as optional post-hoc annotation, not training input -
  the cited page's own language argues against 'Partial'.

Twenty-nine cells were checked and confirmed accurate as already written.

One cell is deliberately left unresolved rather than silently rewritten:
NetMHCpan's 'Multi-virus support' (Pan-pathogen). Neither primary source
discusses peptide biological origin or uses the term 'pan-pathogen' - the
paper's only pan-specificity sentence supports MHC-molecule breadth, a
different claim. Flagged in a README footnote and in claims_register.md D34
pending a maintainer ruling, per rule 3 (when the source cannot be read, the
claim does not get made) - here the *right* replacement claim, not just the
current one, could not be sourced either.

A verification process defect is recorded in D34 because it is exactly the
class docs/claims_register.md and .claude/rules/third-party-claims.md exist
to catch: a first research pass's raw output listed pVACtools's 'Pan-allele
training' current-state field as SESTRAV's OWN column text, a transcription
error rather than a finding about pVACtools. That row was excluded entirely
rather than corrected from a guess - pVACtools's pan-allele cell is unchanged
and unverified by this pass. A second, adversarial verification pass also
caught and rejected two citations containing a fabricated or misattributed
quotation attributed to NetMHCpan's own paper, which is the direct reason
the Multi-virus cell above is unresolved rather than published on the first
pass's citation.

Full source list and per-cell citations: docs/claims_register.md D34.

Verified: full integrity harness produces byte-identical 134 PASS / 0 WARN /
18 FAIL / 7 SKIP on this branch and on bare origin/main - zero new findings
introduced, confirmed by running the harness both with and without this
diff. Doc line citation and commit-reference gates both pass; no banned
typographic character in any added line.

Signed-off-by: Gavin Borges <gavinmborges1104@gmail.com>
@Gavin-Borges
Gavin-Borges merged commit fce43b5 into main Aug 26, 2026
20 checks passed
@Gavin-Borges
Gavin-Borges deleted the docs/readme-competitor-table-citations-b4-l1 branch August 26, 2026 02:59
Gavin-Borges added a commit that referenced this pull request Aug 26, 2026
Resolves the CHANGELOG.md conflict in the [Unreleased] 'Fixed' section, where
both sides appended entries: LRF-1's stage-4 range guard on this branch, and the
D7 provenance-digest, B3 Zenodo-checksum, branch-protection, B1 leave-one-out and
fuzzing.yml entries that landed on main via PR #288 and #291. Both blocks are
kept; nothing is dropped.

Verified after resolution: the branch differs from main by exactly the three
LRF-1 files (CHANGELOG.md, functions/stage4_immunogenicity_scoring.py,
tests/test_stage4_scoring.py), +75 lines and no deletions, so no content that
arrived on main was lost in the merge. No conflict markers and no banned
non-ASCII characters remain in CHANGELOG.md.

Signed-off-by: Gavin Borges <gavinmborges1104@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant