Skip to content

Retain fSuSiE curves and bands while storing each joint fit once - #600

Open
hsun3163 wants to merge 2 commits into
StatFunGen:mainfrom
hsun3163:fix/fsusie-retain-curves
Open

hsun3163 wants to merge 2 commits into
StatFunGen:mainfrom
hsun3163:fix/fsusie-retain-curves

Conversation

@hsun3163

@hsun3163 hsun3163 commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Functional SuSiE already fits all probes in a region jointly, but the joint engine post-processes and stores that same fit once per probe. Each entry carries the full variants-by-features coefficient matrix. This can make the serialized result grow far beyond the memory used to fit the model: the motivating regional output was approximately 277 GiB compressed. At the same time, default trimming removes the functional curves and uncertainty needed to plot the result.

This change keeps one ordinary trimmed fSuSiE entry per joint group, retains its functional summaries, and preserves the existing inputs for colocalization.

Joint fSuSiE output

R/jointEngine.R continues to fit the complete X/Y matrices once. It calls the existing post-processing function for r = 1L and returns list(entry), instead of post-processing the same joint fit once for every trait. All trait names and positions are recorded on the saved fit for functional export.

This fixes duplication in the result assembly, not repeated model fitting. It also avoids repeated post-processing. Smaller regions can complete and serialize with the old code; many probes and a large coefficient matrix expose the storage cost. Ordinary R list sharing during fitting does not make repeated serialized payloads free.

Output-contract change: the retained row has the first trait's identity, conditional summary, and optional CV slice. Separate conditional summaries for the remaining probes are no longer emitted. Their names/positions and the full joint functional curves are retained. This does not assume that probe-specific conditional summaries are identical. PC SuSiE rows follow their existing path.

Trimming and functional accessors

The fSuSiE branch of trimFinemappingFit() retains fitted_func, cred_band, outing_grid, cs, csd_X, and lfsr_func, alongside the previously retained coef. The base trimmer still supplies alpha, lbf_variable, credible sets, and the other colocalization summaries. Wavelet matrices and the full untrimmed object are discarded as before.

The existing native band calculation runs while reconstruction fields are available. This is necessary even when cred_band exists: fsusieR 0.2.97 with post_processing = "none" can initialize it with zero-filled placeholder matrices. After trimming, the existing curve/affected-region accessors reuse the saved bands. Previously these accessors required wavelet fields, so a successfully written trimmed fit could not provide its functional curve summaries. This is a retention/interface defect, independent of whether TI is enabled.

No new storage class, shared environment, package injection, or trim flag is introduced. SuSiE, SuSiE-RSS, and mvSuSiE fitting/trimming branches are unchanged.

Validation

The equivalent 0.6.12 backport, 7dde6d3, was installed and tested in the existing analysis environment:

  • jointEngine, fineMappingWrappers, fsusieAccessors, and colocPipeline test files passed. Two missing-package tests skipped because those packages were installed; one existing zero-variance-column warning was expected.
  • Added regression coverage for both TI and none: trimmed-fit serialization retains curves/bands, affected-region accessors work, log Bayes factors agree with the untrimmed fit, and coloc::coloc.bf_bf runs successfully.
  • The joint-engine regression checks that the fitter receives all traits while producing one entry with all trait names/positions.
  • A canonical xQTL SoS fixture run with TI and one PC per context produced a nonempty combined effect table and a queryable tabix index.

This PR ports the same change to current upstream's FineMappingRow and joint-group accessors. Follow-up validation passed the focused fineMappingRow tests and the canonical xQTL SoS export fixture against the patched 0.8.2 package in an isolated R 4.5 library. The actual loaded package version and library path were verified. Old RDS files whose curves were already discarded require refitting to obtain those curves.

The existing pipeline-level expectations are deliberately retained: they still expect two per-probe rows and therefore expose the intended one-row output-contract change as five assertion failures. They have not been rewritten to make CI green. The existing CI test command now sets stop_on_failure = TRUE; previously devtools::test() printed failed assertions while exiting successfully on macOS/ARM, which skip the separate Linux coverage step.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant