Skip to content

Add a ppm-scale consensus merge as an option - #35

Open
percolator wants to merge 1 commit into
masterfrom
ppm-consensus-merge
Open

percolator wants to merge 1 commit into
masterfrom
ppm-consensus-merge

Conversation

@percolator

@percolator percolator commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

The problem

The consensus merge comes from MS-Cluster (MSClusterMerge), and so do its tolerances: peaks within 0.34 Th are joined inside each spectrum, then the pooled peaks are joined again at 0.22 Th. Those were written for ion-trap data. On Orbitrap fragment spectra, replicate peaks agree to about 1.4 ppm, while 0.22 Th is 440 ppm at m/z 500.

Each join moves an intensity-weighted centroid, so the surviving peak can land where no member spectrum recorded anything. Measured on PXD002370, the UPS1-in-yeast set of the Quandenser paper, nine Q Exactive runs:

measurement value
replicate fragment m/z agreement 1.39 ppm (robust sd)
centroids absorbed by the within-spectrum join 17.7%
consensus peaks >10 ppm from any peak of any member 26.5%
same, among the 20 most intense 24.5%

A quarter of the strongest peaks in a consensus spectrum are at an m/z nothing measured. Charge-3 and charge-4 fragment isotopes, 0.33 and 0.25 Th apart, fall inside the join window.

What this adds

PpmConsensusMerge, off by default:

  • peaks go on a ln(m/z) scale, so one tolerance holds at every mass;
  • two peaks of the same spectrum are never joined: the instrument resolved them, so they are two fragments;
  • the pooled peaks are cut where consecutive ones lie more than numSigmas_ * ppmSigma_ apart, and any group that would still hold two peaks from one spectrum is segmented by dynamic programming over m/z, minimising within-group scatter over the single-peak variance plus a constant per group;
  • the binomial occurrence weight of the existing merge is kept, since it is orthogonal to all of the above and works well.

No new dependencies: STL, plus the existing MZIntensityPair and SpectrumHandler, reusing PeakWeightTable from MSClusterMerge.h.

Interface

flag default meaning
--consensusMethod thomson tolerance scale for merging peaks; ppm selects the new merge. mscluster is accepted as a synonym of thomson
--consensusPpmSigma 2.0 fragment m/z scatter between two spectra of a cluster, in ppm
--consensusMaxPeaks 160 most intense peaks to keep; 0 keeps all

On the ppm path the peak lists are not passed through binMZIntensityPairs, since that is the within-spectrum join the method exists to avoid.

Evidence

A 3,000-cluster sample spanning the cluster-size range, from a PXD002370 run:

check result
default path unchanged peak lists byte-identical to the previous output on all 1,427 shared clusters
consensus peaks >10 ppm from every member peak thomson 21.8%, ppm 0.0%
--consensusMaxPeaks 0 1.34 M peaks against 353 k capped
unit tests maracluster unit-test passes, including a new case covering the ppm grouping, the charge-4 isotope pair the current tolerances collapse, the same-spectrum split and the occurrence weight

Runtime is unchanged, about 42 s for the sample either way, most of it reading mzML.

In a separate evaluation on the full data set (Tide + Percolator, 1% FDR, high-resolution fragment bins), ppm grouping gave +3.0% PSMs on its own and dropping the peak cap +5.2%; an entrapment database showed the share of entrapment hits flat, so the gain is not FDR inflation.

Identifications

Both merges run over one clustering of PXD002370 (101,540 consensus spectra each), searched with Crux Tide against UPS1 + reviewed yeast and rescored with Percolator, 20 ppm precursor window, 0.02 Th fragment bins, 1% FDR:

consensus PSMs clusters peptides UPS1 sequences
thomson (default) 25,761 22,035 11,845 315
ppm, --consensusMaxPeaks 0 28,403 23,437 12,432 327
+10.3% +6.4% +5.0% +12

Same clusters and same precursors, so the difference is the merge and the peak cap alone.

How it was tested

Built on this branch's base with MaRaCluster's own admin/builders/install_proteowizard.sh toolchain, on Ubuntu 22.04 with gcc 11. The unit tests pass and the sample numbers above reproduce byte-identically there and on the older toolchain this work was developed against.

One thing to note for anyone building master today: cmake/FindProteoWizard.cmake does not ask for zstd, but current ProteoWizard's libpwiz_data_msdata_core.a references ZSTD_compress, ZSTD_getErrorCode and friends, so the link fails with undefined references. It is unrelated to this change and I worked around it with -DCMAKE_CXX_STANDARD_LIBRARIES="-Wl,--start-group -L<tools>/lib -l:libzstd-gcc11-mt-s.a -l:libzlib.a -Wl,--end-group". Happy to send that as a separate PR: install_proteowizard.sh makes plain-name symlinks for the Boost components only, so adding one for zstd and listing it in FindProteoWizard.cmake would fix it properly.

🤖 Generated with Claude Code

The consensus merge comes from MS-Cluster (MSClusterMerge), and so do its
tolerances: 0.34 Th within a spectrum and 0.22 Th across a cluster. They were
written for ion-trap data and are two orders of magnitude wider than the few
ppm at which replicate fragments agree on Orbitrap data. Each join moves an
intensity-weighted centroid, so a consensus peak can land where no member
spectrum recorded one: 21.8% of peaks on a PXD002370 sample, 20.7% of the 20
most intense.

The new merge groups peaks on a ln(m/z) scale, never joins two peaks of the
same spectrum, and segments a group that would hold two peaks from one
spectrum by dynamic programming over m/z. The binomial occurrence weight is
kept. On the same sample it invents no peaks.

Off by default: --consensusMethod ppm selects it, --consensusPpmSigma and
--consensusMaxPeaks tune it. The default path is byte-identical to before on
all 1,427 clusters checked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
percolator added a commit to statisticalbiotechnology/quandenser-again that referenced this pull request Sep 22, 2026
MaRaCluster's consensus merge inherits MS-Cluster's fixed Thomson tolerances,
which on Orbitrap fragment spectra are two orders of magnitude wider than the
few ppm at which replicate peaks agree, and which invent peaks: 16.3% of the
consensus peaks of fraction 9 of PXD004863 lie more than 10 ppm from any peak
of any member spectrum. statisticalbiotechnology/maracluster#35 adds a merge
that groups peaks on a ppm scale instead; with it that share is 0.0%, over the
same clusters and the same 10,545 consensus spectra.

Three options are forwarded to MaRaCluster verbatim: --consensus-method,
--consensus-ppm-tol and --consensus-max-peaks. The default is unchanged, and
the default code path in MaRaCluster is byte-identical to before.

The submodule moves to ppm-consensus-merge-a7180b0, that change backported
onto the commit pinned here, so the container still builds against the pinned
ProteoWizard. Once #35 is merged the submodule should move to upstream master,
which needs the container's ProteoWizard step reworked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D7vADr6Z93sRBvfNGHUwW6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant