Cardinal changes

CHANGELOG.md

@@ -20,11 +20,40 @@ and this project adheres to [Semantic Versioning][].
   `tl.external.scores_to_gmt`/`scores_to_covar`, `tl.external.compute_escore`, `JointNMFWrapper`)
 - New "DonorData basics" tutorial: building a `DonorData` from your own genotype/expression
   data (including loading genotypes via `read_sgkit_zarr`/`stream_pgen_to_zarr`), how donor
-  syncing works, selecting subsets, aggregating, and saving/loading — no analysis, just the
+  syncing works, selecting subsets, aggregating, and saving/loading. No analysis, just the
   data structure itself
+- `tl.external.build_known_cis_eqtls_from_tensorqtl`: build a known-cis-eQTL annotation
+  (variant x gene, binary) from a completed TensorQTL nominal cis-scan, for use as a
+  fine-mapping prior
 
 ### Fixed
 
+- `DonorData.copy()` built a genuinely new object but always copied `_G`/`_C`
+  regardless of whether they were views, unlike its previous behavior; reverted to
+  only copying `_G`/`_C` when they're actually views (mutating `self` and returning
+  it, as before)
+- `io.read_zarr_dd` eagerly materialized `G`/`C` on load, unlike `io.read_pgen_zarr`
+  (which already kept a dense `X` Dask-backed); it now uses the same lazy read path,
+  so selecting a subset (e.g. one cell type of `C`) never touches the rest of the data
+- `DonorData.write_zarr_dd` picked its own chunk shape for a dense `X` without regard
+  to any chunking the input already had (confirmed: it disregarded a Dask array's own
+  `.chunks` entirely), which for a genome-scale `X` could pick a shape badly misaligned
+  with the data's on-disk layout and make the write far slower than necessary; `X` is
+  now written separately with sane chunking (a Dask array's own chunks, or capped at
+  4096 per axis for a plain array), overridable via `write_zarr_dd`'s new `x_chunks`
+  argument; the same fix now also applies to any dense `layers` entry (previously only
+  `X` was chunked sanely, so a dense layer the same shape as `X` (e.g. raw counts kept
+  alongside a normalized `X`) still got the pathological chunking)
+- `io.read_zarr_dd`'s lazy read only kept `X` Dask-backed, not a dense `layers` entry,
+  so a dense layer was still eagerly materialized in full on load (and could exhaust
+  memory for a genome-scale one); `layers` is now read lazily the same way `X` is
+- `io._pgen._read_pvar` only mapped a `.pvar`'s columns to cellink's canonical
+  `chrom`/`pos`/`snp_id`/`a0`/`a1` names when a `#CHROM` header line was present;
+  a headerless `.pvar` now gets the same canonical mapping, assuming the standard
+  PLINK2 column order, instead of being left with unnamed columns
+- `io.from_sgkit_dataset` didn't cast `chrom`/`a0`/`a1` to `str`, unlike the pgen
+  reader, which could leave a numeric `chrom` dtype that breaks a downstream string
+  membership check (e.g. in tensorQTL)
 - `pp.log_transform` and `pp.normalize` no longer mutate the input `AnnData` when
   called with `inplace=False`
 - `Skat` and `StructLMM` were unusable (missing `limix-core` imports); both now work,
@@ -41,6 +70,24 @@ and this project adheres to [Semantic Versioning][].
   contributing guide
 - A stray unanchored `data` entry in `.gitignore` silently hid any new file added
   under `tests/data/` or `docs/tutorials/data/`; scoped it to the latter only
+- `io.to_plink` crashed (`TypeError: NDFrame.to_csv() got an unexpected keyword
+  argument 'line_terminator'`) with `pandas-plink` 2.2.9 on a modern pandas, which
+  renamed/removed that `to_csv` kwarg; bumped the minimum `pandas-plink` version to
+  2.3.0, which uses the current `lineterminator` kwarg internally
+
+### Known limitations
+
+- Reading a sparse (`csr`/`csc`) `X` or layer back from a Zarr v3 store is slow:
+  confirmed via two independent hand-built-matrix tests that plain
+  `anndata.io.read_elem` alone (no cellink code involved) takes ~19s for a
+  5M-nnz array that writes in ~0.2s, and `DonorData.write_zarr_dd`/`read_zarr_dd`
+  add only ~9% on top of that baseline, so this is inherent to AnnData's
+  sparse Zarr v3 read path in this environment, not a `DonorData`-specific
+  issue. Not yet root-caused further, and not currently fixed, since no real
+  data in this codebase's own pipelines uses a sparse Zarr layer today (dense
+  `X`/layers only; the sparse `rare`-stratum genotype is read from PGEN
+  directly, never persisted to Zarr). Revisit if/when a real sparse Zarr
+  layer is added.
 
 ### Changed
 

README.md

@@ -1,7 +1,11 @@
+<!-- logo-image-start -->
+
 <p align="center">
   <img src="docs/_static/img/cellink-logo.svg" alt="cellink" width="360">
 </p>
 
+<!-- logo-image-end -->
+
 [![Build](https://github.com/theislab/cellink/actions/workflows/build.yaml/badge.svg)](https://github.com/theislab/cellink/actions/workflows/build.yaml/badge.svg)
 [![License](https://img.shields.io/github/license/theislab/cellink)](https://opensource.org/licenses/Apache2.0)
 [![Read the Docs](https://img.shields.io/readthedocs/cellink/latest.svg?label=Read%20the%20Docs)](https://cellink-docs.readthedocs.io/)

docs/api/tl.md

@@ -19,4 +19,6 @@
     tl.beta_weighting
     tl.subset_genomic_region
     tl.subset_gene
+    tl.coloc_abf
+    tl.coloc_susie
 ```

docs/api/tl_external.md

@@ -2,7 +2,7 @@
 
 Wrappers around external genetics tools. Most of these shell out to a command-line
 tool that has to be installed separately (e.g. PLINK, MAGMA, LDSC, TensorQTL,
-SAIGE-QTL) — see each tool's own documentation for installation instructions.
+SAIGE-QTL). See each tool's own documentation for installation instructions.
 
 ### eQTL mapping (jaxQTL / TensorQTL)
 

docs/contributing.md

@@ -52,7 +52,7 @@ When you're touching one of these wrappers, install just the extra(s) you need,
 `pip install -e ".[pgen,test]"`.
 
 A few (MAGMA, LDSC, PLINK/PLINK2, TensorQTL's CLI mode, SAIGE-QTL) additionally shell
-out to an external binary that isn't distributed on PyPI/conda at all — tests exercising
+out to an external binary that isn't distributed on PyPI/conda at all. Tests exercising
 those either skip via `pytest.importorskip`/an explicit binary check, or aren't run in
 CI and are only expected to pass in an environment where the tool is installed
 separately. Don't assume the base test env in CI covers every `tl.external` module.
@@ -96,7 +96,7 @@ Consider enabling this option for [ruff][ruff-editors] and [prettier][prettier-e
 ## Writing tests
 
 This package uses [pytest][] for automated testing. Please add a test for every
-function you add or change — see scanpy's {doc}`scanpy:dev/testing` guide for general
+function you add or change. See scanpy's {doc}`scanpy:dev/testing` guide for general
 pytest conventions. Small synthetic-data tests (`tests/conftest.py` has `adata`/`gdata`
 fixtures built from `cellink._core.dummy_data`) are preferred over ones that depend on
 `tests/data/*` fixtures or network access; add a real assertion, not just a call that
@@ -197,18 +197,18 @@ Public functions/classes are exported from each submodule's `__init__.py` (and l
 its `__all__`), and separately listed again in the matching `docs/api/*.md` page
 (`donordata.md`, `pp.md`, `io.md`, `tl.md`, `tl_external.md`, `pl.md`, `ml.md`, `at.md`,
 `utils.md`, `resources.md`, `cli.md`). These are two separate, manually-kept-in-sync
-lists — nothing enforces that they match. When you add, rename, or remove a public
+lists. Nothing enforces that they match. When you add, rename, or remove a public
 function:
 
 1. Add it to the submodule's `__init__.py` import and `__all__` (or remove it from both).
 2. Add/update the corresponding `docs/api/*.md` entry.
-3. If it's a new tutorial notebook, add it to `docs/tutorials/index.md`'s toctree — a
+3. If it's a new tutorial notebook, add it to `docs/tutorials/index.md`'s toctree. A
    notebook that exists under `docs/tutorials/` but isn't listed there won't show up in
    the rendered docs navigation.
 
 `ruff`'s `RUF100`/`F822` checks catch a stale `__all__` (a name listed that no longer
 exists), but nothing currently catches the reverse (a public name that exists but was
-never added to `__all__`/the API docs) or a tutorial notebook missing from the toctree —
+never added to `__all__`/the API docs) or a tutorial notebook missing from the toctree,
 worth double-checking by eye.
 
 [sphinx]: https://www.sphinx-doc.org/en/master/
@@ -222,7 +222,7 @@ worth double-checking by eye.
 
 The documentation is set-up to render jupyter notebooks stored in the `docs/tutorials` directory using [myst-nb][].
 Currently, only notebooks in `.ipynb` format are supported that will be included with both their input and output cells.
-It is your responsibility to update and re-run the notebook whenever necessary — notebooks aren't executed as
+It is your responsibility to update and re-run the notebook whenever necessary. Notebooks aren't executed as
 part of CI, so a stale output cell showing an old error/result won't be caught automatically.
 
 If you are interested in automatically running notebooks as part of the continuous integration,

docs/index.md

@@ -1,4 +1,15 @@
 ```{include} ../README.md
+:end-before: '<!-- logo-image-start -->'
+```
+
+```{image} _static/img/cellink-logo.svg
+:align: center
+:width: 360px
+:alt: cellink
+```
+
+```{include} ../README.md
+:start-after: '<!-- logo-image-end -->'
 :end-before: '<!-- schematic-image-start -->'
 ```
 

docs/tutorials/burden_testing.ipynb

@@ -8721,7 +8721,7 @@
    "source": [
     "## Step 2: Burden Testing\n",
     "\n",
-    "We now perform burden tests, which evaluate whether the **aggregate effect** of rare variants within a gene is associated with a phenotype — in this case, gene expression in CD8 Naive cells. We use different weighting schemes allow us to prioritize variants differently based on functional scores or allele frequency, including:\n",
+    "We now perform burden tests, which evaluate whether the **aggregate effect** of rare variants within a gene is associated with a phenotype, in this case gene expression in CD8 Naive cells. We use different weighting schemes allow us to prioritize variants differently based on functional scores or allele frequency, including:\n",
     "- **CADD_RAW**: Raw CADD scores.\n",
     "- **maf_beta**: Beta weighting based on minor allele frequency (MAF).\n",
     "- **tss_distance**: Distance to the transcription start site (TSS).\n",
@@ -8746,7 +8746,7 @@
    "metadata": {},
    "source": [
     "### Filtering for Rare Variants\n",
-    "Burden tests focus on rare variants—typically those with minor allele frequency (MAF) below 0.01 or 0.05. We filter our genotype data accordingly to isolate variants in this frequency range."
+    "Burden tests focus on rare variants, typically those with minor allele frequency (MAF) below 0.01 or 0.05. We filter our genotype data accordingly to isolate variants in this frequency range."
    ]
   },
   {