biospin · kakyungkim · Aug 11, 2026 · Aug 11, 2026
diff --git a/pipeline/hspc-velocity-benchmark/manuscript/draft_v2.md b/pipeline/hspc-velocity-benchmark/manuscript/draft_v2.md
@@ -116,7 +116,7 @@ Per-gene kinetic parameters are not what velocity is mainly used for, so we exte
 
 Two controls frame these numbers, and both are MultiVelo's, because it is the only arm whose refits were retained. Refitting MultiVelo on resampled cells reproduces its own matrix at mean-centred cosine +0.872 (six refits, range +0.826 to +0.887; sign agreement 78.6%), so the measure does detect agreement where agreement exists. The second control destroys the chromatin channel by permuting the ATAC rows (cells) within each lineage, which leaves the RNA channel and the lineage-level chromatin structure intact. Our first version of this second comparison was not like-for-like, and we correct it here: the refit ceiling was computed on 15,315 resampled cells while the shuffled fit used all 21,878, so the two arms did not share a cell set, and the resulting shuffled value (+0.838) sat inside the ceiling range only under that mismatch. Repeating the shuffle on the *same* resampled cell sets S_b used by three of the refits — identical hyperparameters, gene set and shuffle protocol, with the two arms' cell-name vectors asserted equal at both fit and analysis time — puts the shuffled matrix below the intact refit range in all three pairs (+0.784, +0.813 and +0.810 against an intact range of +0.826 to +0.887), with non-overlapping cell-bootstrap intervals in three of three (`results/velocity_matrix_paired_shuffle.md`). We therefore withdraw our earlier statement that chromatin is inert in this matrix; it can no longer be asserted.
 
-What replaces it is bounded rather than absent. The paired differences (Δ = A − B) are small and all positive, but their magnitude is not fixed to a single value: it varies with the shuffle draw, spanning roughly two- to three-fold within a single cell set, so no one draw is a representative size. Across draws Δ sits at a single-digit percentage of the +0.872 ceiling (of order 7%), well below the disagreement that method choice produces on the same measure (mean-centred cosine −0.530 to +0.131 across method pairs). Whether any of that residue is chromatin rather than refitting was initially unresolved, because no same-cell, same-configuration rerun null existed for MultiVelo; that gap has since been closed, and it closes towards a contribution rather than towards noise. Refitting on the same cell set S_b with chromatin intact, varying nothing but the worker count, reproduces the previous fit exactly — the two same-day fits are bit-identical and the run-to-run difference on the audit's own metric is Δ_rr = 0.000000, with a degenerate cell-bootstrap interval, for both cell sets measured (`results/velocity_matrix_runtorun_null.md`). Against the archived fits the only departure is a single gene (*LRIG1*): a 4×10⁻¹⁶ rounding difference in one cell set and, in the other, an alternative solution of near-equal loss (switch time 65.0 versus 66.9) that moves the per-cell mean-centred cosine by at most 2×10⁻⁶. Because the fit is deterministic under the conditions we use, the paired Δ carries no refit-noise component to subtract, and the preregistered condition for a positive statement — the upper interval of |Δ_rr| below the lower interval of Δ_paired — holds in both cases (0.000000 against 0.0792 and 0.0608). We therefore state, for MultiVelo and for this matrix, that destroying the chromatin channel does move the output and that the movement is not an artefact of refitting. Three limits keep the statement small. Only two of the three pairs have a matched rerun null, and the pair with the smallest original draw is not among them. Repeating the shuffle under four different draws per cell set (twelve fits in total) separates what survives resampling from what does not: the sign survives — all twelve draws give Δ>0 with intervals excluding zero — while the magnitude does not, since the spread across draws exceeds the median in one cell set (range 0.043 against a median of 0.031) and the single draw we had reported there (+0.063) turns out to be the largest of the four (`results/velocity_matrix_shuffle_seed_variability.md`). This dispersion verdict is sealed on the range, and a variance-based dispersion measure would instead pass; either way the substantive point — a two- to three-fold spread of magnitude across draws — is metric-independent. We therefore state the direction and decline to pin the size to a number; four draws per cell set is itself few, so the spread is coarsely estimated. And the effect stays an order of magnitude below what method choice produces on the same measure, so "chromatin contributes here" is not "chromatin makes this matrix reliable". This matrix-level movement is also not in tension with the lag result: the same class of ATAC shuffle does not perturb the priming-marker lags more than a bulk shuffle (Fig. 2, chromatin does not set the lag), yet here it moves the cell×gene velocity matrix; the two findings concern different targets and are consistent rather than contradictory. We report these comparisons on mean-centred cosine because the raw value is partly determined by a direction common to all cells (the mean vector accounts for 12.9–37.4% of squared row norm, depending on the arm); centring is a post-hoc diagnostic and is not part of the sealed metric list, and the preregistered verdict above rests on the raw metric. One further limit is load-bearing: reading the cross-method values as genuine disagreement rather than arm-internal instability is licensed only for MultiVelo, which has that control. The three pairs closest to zero all involve MoFlow, a stochastic deep model whose same-configuration rerun stability was never established (its own original-versus-shuffled value, +0.113, is uninterpretable for the same reason), so for those pairs disagreement and instability are not separable.
+What replaces it is bounded rather than absent. The paired differences (Δ = A − B) are small and all positive, but their magnitude is not fixed to a single value: it varies with the shuffle draw, spanning roughly two- to three-fold within a single cell set, so no one draw is a representative size. Across draws Δ sits at a single-digit percentage of the +0.872 ceiling (of order 7%), well below the disagreement that method choice produces on the same measure (mean-centred cosine −0.530 to +0.131 across method pairs). Whether any of that residue is chromatin rather than refitting was initially unresolved, because no same-cell, same-configuration rerun null existed for MultiVelo; that gap has since been closed, and it closes towards a contribution rather than towards noise. Refitting on the same cell set S_b with chromatin intact, varying nothing but the worker count, reproduces the previous fit exactly — the two same-day fits are bit-identical and the run-to-run difference on the audit's own metric is Δ_rr = 0.000000, with a degenerate cell-bootstrap interval, for both cell sets measured (`results/velocity_matrix_runtorun_null.md`). Against the archived fits the only departure is a single gene (*LRIG1*): a 4×10⁻¹⁶ rounding difference in one cell set and, in the other, an alternative solution of near-equal loss (switch time 65.0 versus 66.9) that moves the per-cell mean-centred cosine by at most 2×10⁻⁶. Because the fit is deterministic under the conditions we use, the paired Δ carries no refit-noise component to subtract, and the preregistered condition for a positive statement — the upper interval of |Δ_rr| below the lower interval of Δ_paired — holds in both cases (0.000000 against 0.0792 and 0.0608). We therefore state, for MultiVelo and for this matrix, that destroying the chromatin channel does move the output and that the movement is not an artefact of refitting. Three limits keep the statement small. Only two of the three pairs have a matched rerun null, and the pair with the smallest original draw is not among them. Repeating the shuffle under four different draws per cell set (twelve fits in total) separates what survives resampling from what does not: the sign survives — all twelve draws give Δ>0 with intervals excluding zero — while the magnitude does not, since the spread across draws exceeds the median in one cell set (range 0.043 against a median of 0.031) and the single draw we had reported there (+0.063) turns out to be the largest of the four (`results/velocity_matrix_shuffle_seed_variability.md`). This dispersion verdict is sealed on the range, and a variance-based dispersion measure would instead pass; either way the substantive point — a two- to three-fold spread of magnitude across draws — is metric-independent. We therefore state the direction and decline to pin the size to a number; four draws per cell set is itself few, so the spread is coarsely estimated. And the effect stays an order of magnitude below what method choice produces on the same measure, so "chromatin contributes here" is not "chromatin makes this matrix reliable". This matrix-level movement is also not in tension with the lag result: the same class of ATAC shuffle does not perturb the priming-marker lags more than a bulk shuffle (Fig. 2, chromatin does not set the lag), yet here it moves the cell×gene velocity matrix; the two findings concern different targets and are consistent rather than contradictory. We report these comparisons on mean-centred cosine because the raw value is partly determined by a direction common to all cells (the mean vector accounts for 12.9–37.4% of squared row norm, depending on the arm); centring is a post-hoc diagnostic and is not part of the sealed metric list, and the preregistered verdict above rests on the raw metric. One further limit is load-bearing: reading the cross-method values as genuine disagreement rather than arm-internal instability is licensed only for MultiVelo, which has that control. That licence has since extended to MoFlow. A matched same-configuration rerun, two independent refits on the identical cell and gene set, reproduces MoFlow's original matrix at mean-centred cosine +0.9999 [+0.9999, +0.9999] (run-versus-run, +1.0000), a ceiling far above the 0.0118 maximum among the three near-zero pairs that involve it (`results/moflow_runtorun_null.md`). Those near-zero values therefore read as genuine near-zero cross-method agreement rather than MoFlow's own instability, and the same ceiling places MoFlow's original-versus-shuffled value (+0.113) well clear of run-to-run noise.
 
 Read within those limits, the largest mean-centred agreement anywhere in the comparison was between MultiVelo and the RNA-only scVelo floor (+0.583) rather than between two multiome methods — though this is not a family property, since the corresponding values for CRAK-Velo, MoFlow and MultiVeloVAE were +0.260, −0.004 and −0.292. MultiVelo and MultiVeloVAE assigned systematically opposite directions to the same cells (mean-centred −0.500); whether that is a substantive disagreement or an undocumented difference in sign or parameterisation convention cannot be settled by this design, but either way an analyst who swaps one output for the other without checking obtains opposing directions. Both contrasts failed on the metric and thresholds sealed before the fitted matrices were read (Additional file 12): multiome pairs did not agree more than the RNA-only baseline, and destroying chromatin did not collapse the matrix, although the paired comparison above shows that it does move it a little. The matrix therefore reproduces across methods no better than the per-gene parameters did. It also sits alongside the general benchmarks, which report low cross-method agreement of transition vectors in RNA-only settings (A1<0.3 across the twelve methods compared) [25].
 
@@ -204,7 +204,7 @@ All cross-method and cross-dataset concordances were Spearman rank correlations.
 
 ### Cell-level velocity-matrix audit
 
-Definitions, exclusion rules and falsification criteria were sealed before the fitted matrices were read. Cells and genes were matched by name across arms; all five arms share the same 21,878 cells, and the 505 genes common to all five were reduced to 354 after excluding any gene with a missing (scVelo leaves 151 genes with unrecovered dynamics) or constant velocity in any arm. Only spliced velocity was compared (MultiVelo and MoFlow `velo_s`, CRAK-Velo and scVelo `velocity`, MultiVeloVAE `vae_velocity`); chromatin velocity layers were never mixed in. Agreement was the per-cell cosine similarity across genes, reported against a cell-shuffled null (method A's cell *i* against method B's randomly permuted cell *j*) because a globally shared velocity direction inflates the raw value; medians carry a cell bootstrap 95% CI (B=10³, seed 20260719). Because raw cosine is partly determined by a direction common to all cells (the mean vector accounts for 12.9–37.4% of squared row norm depending on the arm), we also report the mean-centred cosine, which measures agreement of the cell-specific deviations; centring is applied without scaling so that sign is preserved. Per-(cell,gene) sign agreement excludes entries that are exactly zero in either arm, following the same undetermined-direction convention used for the lag. Two reference points frame the cross-method values: a reproducibility ceiling from six MultiVelo bootstrap refits on resampled cells, and the ATAC-shuffle arms used for the lag control. Because those two were originally computed on different cell sets, the within-lineage ATAC shuffle was additionally refit on the same three resampled cell sets used by three of the refits, with identical hyperparameters, gene set and shuffle protocol and with the two arms' cell-name vectors asserted equal, giving a paired comparison and paired cell-bootstrap intervals (`results/velocity_matrix_paired_shuffle.md`). The causal reading is restricted to MultiVelo, the only arm with a refit control, and no same-cell same-configuration rerun null was available for either arm.
+Definitions, exclusion rules and falsification criteria were sealed before the fitted matrices were read. Cells and genes were matched by name across arms; all five arms share the same 21,878 cells, and the 505 genes common to all five were reduced to 354 after excluding any gene with a missing (scVelo leaves 151 genes with unrecovered dynamics) or constant velocity in any arm. Only spliced velocity was compared (MultiVelo and MoFlow `velo_s`, CRAK-Velo and scVelo `velocity`, MultiVeloVAE `vae_velocity`); chromatin velocity layers were never mixed in. Agreement was the per-cell cosine similarity across genes, reported against a cell-shuffled null (method A's cell *i* against method B's randomly permuted cell *j*) because a globally shared velocity direction inflates the raw value; medians carry a cell bootstrap 95% CI (B=10³, seed 20260719). Because raw cosine is partly determined by a direction common to all cells (the mean vector accounts for 12.9–37.4% of squared row norm depending on the arm), we also report the mean-centred cosine, which measures agreement of the cell-specific deviations; centring is applied without scaling so that sign is preserved. Per-(cell,gene) sign agreement excludes entries that are exactly zero in either arm, following the same undetermined-direction convention used for the lag. Two reference points frame the cross-method values: a reproducibility ceiling from six MultiVelo bootstrap refits on resampled cells, and the ATAC-shuffle arms used for the lag control. Because those two were originally computed on different cell sets, the within-lineage ATAC shuffle was additionally refit on the same three resampled cell sets used by three of the refits, with identical hyperparameters, gene set and shuffle protocol and with the two arms' cell-name vectors asserted equal, giving a paired comparison and paired cell-bootstrap intervals (`results/velocity_matrix_paired_shuffle.md`). The causal reading is restricted to MultiVelo, the only arm with a cell-bootstrap refit control. A same-cell, same-configuration rerun null (fixed cells and genes, independent refits) was subsequently established for MultiVelo (`results/velocity_matrix_runtorun_null.md`) and, separately, for MoFlow (`results/moflow_runtorun_null.md`); see Results.
 
 ### Enrichment analysis