MANTIS

Multimodal Alignment of Normalized Tissue Sections

Python tools to align H&E and IHC whole-slide images (WSIs) via comparable thumbnails, tissue masks, and 2D mask cross-correlation. A second CLI command correlates MIL attention scores with IHC positivity (e.g. Ki67 or ER) on the mapped WSI.

Quick start

Requires Python ≥ 3.10. Activate your environment, then install the package (editable) so the mantis command is on your PATH:

cd MANTIS
pip install -e .
mantis --help

Optional Poetry workflow (if you prefer it):

poetry env use /path/to/python3.10+
poetry install
poetry run mantis --help

CLI

mantis exposes two subcommands:

Subcommand	Purpose
pipeline	Full H&E ↔ IHC alignment → {output_dir}/{slide_id}.tif
attention-corr	Pearson r between MIL attention and mapped IHC positivity per patch

(pipeline is optional: mantis --he-slide ... still works as shorthand.)

pipeline — alignment

The IHC flag is still named --ki67-slide for historical reasons; pass any comparable IHC pyramid (Ki67, ER, …).

mantis pipeline \
  --he-slide /path/to/365940he.tif \
  --ki67-slide /path/to/365940ki67.tif \
  -o ./mantis_out
# → ./mantis_out/365940he.tif
# → ./mantis_out_work/365940he/  (thumbnails, masks, …)

• Output dir (-o): final mapped WSI only, named {slide_id}.tif (default slide_id = H&E slide stem, e.g. 293892he.tif).
• Work dir (--work-dir = root, default {output-dir}_work): per-slide folder {work-dir}/{slide_id}/ with thumbnails, masks, labels, memmaps, JSON/CSV (safe for parallel jobs).

attention-corr — attention vs mapped IHC

After pipeline (or an existing mapped WSI), compute per-patch IHC positivity (yellow/brown DAB > blue counterstain) vs MIL attention, with both variables z-scored on valid patches before Pearson correlation. Use --intensity-mode luma for the legacy grayscale mean.

Stdout is only the correlation float (no labels), suitable for scripting.

Pass the Trident coords and MIL attention HDF5 paths explicitly:

CORR=$(mantis attention-corr \
  --slide-id 293892he \
  -o ./mantis_out \
  --coords-h5 /path/to/trident/.../features_uni_v2/293892he.h5 \
  --attention-h5 /path/to/results/.../attention_scores/293892he.h5 \
  --patch-size 256)
echo "$CORR"
# e.g. 0.140826

Optional side outputs (silent on stdout). --plot-layout segments_sidebyside implies --exclude-zero-ihc for the printed r and the scatter:

mantis attention-corr \
  --slide-id 293892he \
  --coords-h5 /path/to/coords.h5 \
  --attention-h5 /path/to/attention.h5 \
  --exclude-zero-ihc \
  --plot-layout segments_sidebyside \
  --plot-out ./mantis_out/attention_corr_segments.png \
  --json-out ./mantis_out/attention_corr.json

--plot-out needs matplotlib; features_table() in the Python API needs pandas (neither is a hard install dependency).

Typical mag-corrected paths:

• Coords: {processed_data}/trident(mag_corrected)/20x_256px_0px_overlap/features_uni_v2/{slide_id}.h5
• Attention: {processed_data}/results/er/multi_instance-mag_corrected/heatmaps/uni_v2.abmil/attention_scores/{slide_id}.h5

Pipeline steps

1. Thumbnails — thumb_he.jpg / thumb_ki67.jpg at shared µm/px (thumbnail_meta.json).
2. Segmentation — Otsu masks (mask_he.png, mask_ki67.png).
3. Global align — Coarse + fine cross-correlation (correlation_table.csv).
4. Clusters + refine — DBSCAN, overlap/elbow filters, per-cluster rigid align, WSI export.

Requirements

OpenCV, NumPy, SciPy, tifffile, imagecodecs, h5py (installed automatically via pip install -e .).

CLI options (pipeline)

Option	Default	Description
--he-slide	(required)	H&E pyramid TIFF
--ki67-slide	(required)	IHC pyramid TIFF (Ki67, ER, …)
-o / --output-dir	mantis_out	Final mapped WSI directory
--work-dir	{output-dir}_work	Work root; files go under {work-dir}/{slide-id}/
--slide-id	H&E slide stem	Final TIFF name {slide_id}.tif
--mag-he, --mag-ki67	20	Magnification if mpp not in TIFF
--max-long-edge	2000	Thumbnail long edge (px)
--coarse-step	10	Coarse alignment step
--refine-size	20	Fine search window
--eps, --min-samples	1.5, 10	DBSCAN clustering
--min-overlap-ratio	0.5	HE∩IHC overlap filter
--angle-max, --trans-range	15, 10	Per-cluster search
--export-viz	off	Extra aligned label PNGs

CLI options (attention-corr)

Option	Default	Description
--slide-id	(required)	Slide stem for mapped WSI / work dir
-o / --output-dir	mantis_out	Directory with mapped WSI
--mapped-wsi	{output-dir}/{slide-id}.tif	Override mapped WSI path
--coords-h5	(required)	Trident features HDF5 (coords)
--attention-h5	(required)	MIL attention scores HDF5
--patch-size	256	Patch edge on mapped WSI (px)
--window-size	1	Sliding window side in patch units (1 = per-patch)
--pooling	mean	Window aggregation: mean or max (ignored if window is 1)
--intensity-mode	ihc	ihc = yellow/brown > blue; luma = legacy grayscale
--exclude-zero-ihc	off	Drop IHC intensity == 0 before z-score / Pearson
--work-dir	{output-dir}_work	Work root; files go under {work-dir}/{slide-id}/
--plot-layout	grids	grids or segments_sidebyside
--plot-out	—	Save figure PNG (requires matplotlib)
--json-out	—	Save summary JSON with correlation, n_patches, n_valid

Exit code 0 on success; errors go to stderr.

The RGB memmap under the work dir is created with an exclusive lock and atomic publish (plus a .ok completion marker), so parallel attention-corr jobs for the same slide can safely share one complete file. Full-length leftovers without a marker are rebuilt.

Attention correlation (Python API)

from mantis import MappedSlideAttentionCorrelation

analysis = MappedSlideAttentionCorrelation.from_h5(
    "mantis_out/293892he.tif",
    ".../attention_scores/293892he.h5",
    ".../features_uni_v2/293892he.h5",
    patch_size=256,
    window_size=1,
    pooling="mean",
    intensity_mode="ihc",
    exclude_zero_ihc=False,
    temp_dir="mantis_out_work/293892he",
)
result = analysis.run()
result.correlation       # Pearson r (z-scored IHC vs attention)
result.patch_intensity   # per-patch IHC positivity (yellow/brown > blue)
result.features_table()  # pandas DataFrame (optional; needs pandas)

Coordinate mapping (Python API)

from mantis.coord_map import Ki67ToHEMapper

mapper = Ki67ToHEMapper.from_mantis_dir("mantis_out", slide_id="293892he")
# reads mantis_out_work/293892he/

# Thumbnail pixels (thumb_ki67 → thumb_he)
he_y, he_x, cluster_id, method = mapper.ki67_to_he(ki67_y, ki67_x)

# WSI pyramid level 0
he_y_wsi, he_x_wsi, ki_y_wsi, ki_x_wsi, cid, method = mapper.ki67_to_he_wsi(ki67_y, ki67_x)

Scale: y_wsi = y_thumb * scale_y (per slide in thumbnail_meta.json).

Tests

pip install -e .
pip install pytest
pytest

With Poetry: poetry install --with dev && poetry run pytest.

CI runs on push/PR to main or master (see .github/workflows/tests.yml).