Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
# MkDocs Material site configuration for GenAIRR docs.
#
# Status: scaffold only. Phase 0 of the migration plan in
# `docs/docs_mkdocs_migration_plan.md`. No content has been
# migrated yet — most nav entries below point at placeholder
# files or files that don't exist yet. The contract pins in
# `tests/test_docs_mkdocs_migration_contract.py` verify
# scaffold completeness, not buildability.
# The site is live and built from `site_docs/`. Some deeper pages
# (a handful of concept/validation/cartridge-estimator stubs) are
# still being written; the nav below ships the pages that have
# content. The contract pins in
# `tests/test_docs_mkdocs_migration_contract.py` guard structure.
#
# To build (once Phase 1 installs deps):
# To build:
# pip install -e ".[docs]"
# mkdocs build --strict
#
# Existing live URLs at `mutejester.github.io/GenAIRR/<page>.html`
# are preserved via the `redirects` plugin block (commented out
# below until mkdocs-redirects is installed in Phase 1).
# are preserved via the `redirects` plugin block below.

site_name: GenAIRR
site_description: Synthetic immune-receptor-sequence simulator with absolute ground truth
Expand Down Expand Up @@ -140,7 +138,9 @@ plugins:
# — preserving every live `mutejester.github.io/GenAIRR/<page>.html`.
# 29 entries: one per existing website/*.html page, modulo
# `index.html` which is the homepage itself (`/` already serves it).
"learn.md": "getting-started/index.md"
# NOTE: no 'learn.md' redirect — learn.md is a real nav page
# ("Choose your path"); a redirect here would clobber the built
# learn.html with a stub and make the page unreachable.
"lesson-1.md": "getting-started/lesson-1-vdj-recombination-from-bases-up.md"
"lesson-2.md": "getting-started/lesson-2-the-pipeline-scrubber.md"
"lesson-3.md": "getting-started/lesson-3-s5f-mutation-isnt-uniform.md"
Expand Down Expand Up @@ -197,10 +197,11 @@ nav:
- Core Concepts:
- concepts/index.md
- AIRR Record: concepts/airr-record.md
- Reference Cartridge: concepts/reference-cartridge.md
- Reference Cartridge (concept): concepts/reference-cartridge.md
- Simulation Guides:
- guides/index.md
- The Experiment builder: guides/experiment-builder.md
- Sample only productive sequences: guides/productive.md
- Biology map: guides/biology-map.md
- Recombination + junction biology: guides/recombination-junction.md
- D inversion + receptor revision: guides/recombination-editing.md
Expand Down
4 changes: 2 additions & 2 deletions site_docs/concepts/airr-record.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ comma-separated **tie set** when the engine sampled an
indistinguishable group:

```text
v_call: "IGHV3-23*01"
v_call: "IGHV3-23*01,IGHV3-23*04" ← tie set: two-way ambiguity
v_call: "IGHVF1-G2*01"
v_call: "IGHVF1-G2*01,IGHVF1-G2*02" ← tie set: two-way ambiguity
```

Downstream code that wants the single committed allele should
Expand Down
11 changes: 6 additions & 5 deletions site_docs/concepts/reference-cartridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,16 +70,17 @@ opt-in can mask it.
The set of V / D / J / C alleles available to the simulator. Each
allele carries a name, gene, sequence, segment, and an optional
anchor position (V Cys / J W-or-F codon offset). The catalogue is
exactly what `cfg.add_v_allele(...)` etc. have always populated -
the cartridge concept is unchanged from earlier GenAIRR versions.
read through `cfg.v_alleles` / `cfg.d_alleles` / `cfg.j_alleles`
(dicts keyed by gene) - the cartridge concept is unchanged from
earlier GenAIRR versions.

V alleles can additionally carry **subregion annotations** - the
five canonical IMGT region intervals (`FWR1` / `CDR1` / `FWR2` /
`CDR2` / `FWR3`) that drive per-V-region SHM targeting and per-V-
region mutation counters. When a V allele has a populated IMGT-
gapped sequence, the bridge derives these intervals for you;
otherwise you can supply them explicitly. See
[V-subregion annotations](../cartridges/v-subregion-annotations.md).
[Build a reference cartridge](../guides/build-reference-cartridge.md).

### Rules

Expand Down Expand Up @@ -123,7 +124,7 @@ from GenAIRR import (

cfg.reference_models = ReferenceEmpiricalModels(
allele_usage=AlleleUsageSpec(
v={"IGHV1-2*02": 100.0, "IGHV3-23*01": 60.0},
v={"IGHVF1-G1*01": 100.0, "IGHVF1-G2*01": 60.0},
),
trims={
"V_3": EmpiricalDistributionSpec([(0, 5.0), (1, 3.0), (2, 1.0)]),
Expand Down Expand Up @@ -234,7 +235,7 @@ P-nucleotide lengths) and writes the result into the corresponding
typed empirical-model sub-plane. The build report's
`stages` list captures every estimator's inputs and inferred
distributions for provenance. See
[Cartridge estimators](../cartridges/estimators/index.md) for
[Estimate models from data](../guides/estimate-cartridge-models.md) for
per-estimator detail.

## The manifest
Expand Down
2 changes: 1 addition & 1 deletion site_docs/getting-started/export-results.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ on the TSV/DataFrame; the FASTQ files only carry sequence + quality.
- **[Validate AIRR records](../validation/validate-records.md)** -
confirm every field is internally consistent before downstream
analysis.
- **[The simulation pipeline concept](../concepts/pipeline.md)** -
- **[The Experiment builder](../guides/experiment-builder.md)** -
the mental model behind the DSL and what each pass actually does.
- **[Productive sampling guide](../guides/productive.md)** - what
"productive" means biologically and how the constraint composes
Expand Down
8 changes: 4 additions & 4 deletions site_docs/guides/biology-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,9 @@ experimental design (Experiment).

The reference cartridge carries the immutable biological priors:

- **Allele universe** - `cfg.alleles.v`, `cfg.alleles.d`,
`cfg.alleles.j`, `cfg.alleles.c`. The sequences and metadata
the recombinase has access to.
- **Allele universe** - `cfg.v_alleles`, `cfg.d_alleles`,
`cfg.j_alleles`, `cfg.c_alleles` (dicts keyed by gene). The
sequences and metadata the recombinase has access to.
- **Empirical recombination distributions** -
`cfg.reference_models.allele_usage`,
`cfg.reference_models.trims`,
Expand All @@ -119,7 +119,7 @@ The reference cartridge carries the immutable biological priors:
draws the recombination pass samples from.
- **SHM kernel** - the cartridge's S5F mutability table (used
when `.mutate(model="s5f", ...)` runs).
- **V-subregion annotations** - `cfg.alleles.v[i].subregions`.
- **V-subregion annotations** - `cfg.v_alleles[gene][i].subregions`.
Required for `v_subregion_rates`.
- **Rules / anchors** - `cfg.reference_rules` (V Cys + J anchor
expectations, allowed bases, severity).
Expand Down
4 changes: 2 additions & 2 deletions site_docs/guides/experiment-builder.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ exp.mutate(model="uniform", count=12) # exactly 12 uniform mutations per recor

`end_loss_*prime(length=...)` follows the same `int` / `(low, high)` /
`[(value, weight), ...]` shape. Read [`Tune corruption
rates`](tune-corruption.md) for the per-mechanism shape reference;
rates`](corruption-sequencing.md) for the per-mechanism shape reference;
the API reference page has the formal signatures.

## How `productive_only()` works
Expand Down Expand Up @@ -269,5 +269,5 @@ seed produces the same draws, which is how golden tests work.
| Authoring or tuning a custom reference cartridge | [Reference cartridge](../concepts/reference-cartridge.md) |
| Confirming output integrity post-run | [`validate_records`](../validation/validate-records.md) |
| Per-record AIRR field catalogue | [Your first AIRR record](../getting-started/first-airr-record.md) |
| The mental model behind the DSL | [The simulation pipeline](../concepts/pipeline.md) |
| The mental model behind the DSL | [Core concepts](../concepts/index.md) |
| Every public symbol's signature | [API reference](../reference/index.md) |
7 changes: 7 additions & 0 deletions site_docs/guides/genotype-benchmarking.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,3 +160,10 @@ Because the genotype is planted, every tool is scored identically: recovered
allele set vs `genotype.to_table()` - presence precision/recall, zygosity, and
deletion calls.

## Where to go next

- [Genotypes (overview)](genotype.md) - building genotypes, novel alleles, and
the `to_table()` / `to_tsv()` ground truth you score against.
- [Genotype cohorts](genotype-cohorts.md) - scale this recipe to many donors in
one `run_cohort` call.

8 changes: 8 additions & 0 deletions site_docs/guides/genotype-cohorts.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,11 @@ a cohort is fully reproducible and subjects are independent.
its own carried alleles); clonal forks are not combined with a cohort in this
release.

## Where to go next

- [Genotypes (overview)](genotype.md) - the core model and the `Genotype` builder.
- [Sampling & population priors](genotype-priors.md) - draw the per-subject
genotypes a cohort runs on.
- [Benchmarking genotype inference](genotype-benchmarking.md) - score a tool
against the planted truth; the recipe applies per subject across a cohort.

14 changes: 10 additions & 4 deletions site_docs/guides/genotype.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,11 @@ result[0]["haplotype"] # 0 or 1 - which chromosome this read used
result.genotypes[0].to_table() # ground-truth genotype (per gene, per haplotype)
```

`subject_id` and `haplotype` are on every record; add
`run_records(..., expose_provenance=True)` to also get the
`truth_v_call` / `truth_d_call` / `truth_j_call` columns (the alleles the
engine actually sampled).

## How recombination samples from a genotype

When a genotype is attached, recombination runs a single phased sampling pass per
Expand Down Expand Up @@ -195,8 +200,9 @@ Notes and guard-rails:
- **`complete_from_reference(policy=...)`** fills only genes you haven't touched.
`"homozygous_first_reference"` (default) makes each unspecified gene homozygous
for its first cartridge allele - note this is the *first listed* allele, **not**
a population-frequency-common one (GenAIRR has no frequency prior in this
release; the name says exactly what it does). `"heterozygous_first_two"` uses
a population-frequency-common one (this policy consults no frequency prior; for
frequency-driven genotypes use
[`Genotype.sample`](genotype-priors.md)). `"heterozygous_first_two"` uses
the first two alleles.
- Unknown gene/allele names, NaN/inf chromosome weights, and segments left with no
usable allele are all rejected at build/attach with clear messages.
Expand All @@ -216,8 +222,8 @@ allele-usage model (`reference_models.allele_usage`), aggregated to the gene
level, and scaled by **copy-number dosage** (a duplicated gene recombines
proportionally more often). Cartridges that don't author a typed `allele_usage`
fall back to uniform-over-present-genes (× dosage). See
[Allele usage](v-usage.md) and [Estimate models from data](estimate-cartridge-models.md)
for authoring usage.
[Allele usage](recombination-junction.md#controlling-the-allele-universe) and
[Estimate models from data](estimate-cartridge-models.md) for authoring usage.

### More genotype recipes

Expand Down
90 changes: 81 additions & 9 deletions site_docs/guides/productive.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,83 @@
# Sample only productive sequences

!!! info "Migration placeholder"
This page is a placeholder. Content lands in **Phase 3**
of the migration plan documented in
[`docs/docs_mkdocs_migration_plan.md`](https://github.com/MuteJester/GenAIRR/blob/master/docs/docs_mkdocs_migration_plan.md).

Until then, see the corresponding live page on the existing
documentation site at
[mutejester.github.io/GenAIRR](https://mutejester.github.io/GenAIRR/)
or the source markdown referenced above (if any).
<p class="lead">`productive_only()` constrains V(D)J sampling so the engine only ever
produces <strong>productive</strong> rearrangements - in-frame junction, no stop
codon in the junction, and the conserved V (Cys) and J (Trp/Phe) anchors intact.
It is a chainable, order-independent step on the <code>Experiment</code>.</p>

```python
import GenAIRR as ga

result = (
ga.Experiment.on("human_igh")
.recombine()
.productive_only() # constrain to productive rearrangements
.run_records(n=1000, seed=7)
)
# every record: rec["productive"] is True
```

## What "productive" means here

A rearrangement is productive when its assembled coding sequence could yield a
functional receptor. GenAIRR enforces the **productive triad**:

- the junction is **in-frame** (the V(D)J join preserves the reading frame),
- there is **no stop codon in the junction**, and
- the conserved anchors are intact - the **V cysteine** and the **J
tryptophan/phenylalanine** that bound the CDR3.

## How it works - constrain before propose

`productive_only()` is **not** a reject-and-resample loop that throws away bad
draws after the fact. GenAIRR follows a **constrain-before-propose** discipline:
the constraint narrows the *candidate support at each draw*, so out-of-frame or
stop-bearing candidates are never proposed in the first place. The engine samples
only from the admissible set.

This is why it is efficient and why the productive fraction jumps from the
unconstrained baseline (heavy chains are productive only a minority of the time
by chance - about [18.5% in the demo](../demo.md)) to 100%. It also composes
with the other sampling constraints (e.g.
[`restrict_alleles`](recombination-junction.md#controlling-the-allele-universe)) -
the engine intersects all active constraints into one admissible support, so the
choice of where in the chain you call `productive_only()` doesn't matter.

## Interaction with SHM

The same discipline extends to somatic hypermutation. When `productive_only()` is
in the pipeline, the [mutation pass](shm-targeting.md#interaction-with-productive_only)
never proposes a substitution that would break the productive triad - the
constraint masks the per-site proposal support, so a mutation that would
introduce a junction stop or destroy an anchor simply can't be drawn. `productive`
stays `True`, and `n_mutations` counts only the substitutions that landed and
survived the mask.

## What you get on the record

Every record carries the AIRR `productive` field; with `productive_only()` it is
`True` for all of them. The output is productive *by construction* - you do not
need to filter post-hoc, and [`validate_records`](../validation/validate-records.md)
will confirm the productive invariants hold for every row.

## When to use it

- **Realistic functional repertoires** - most expressed immunoglobulin (BCR)
sequences are productive; `productive_only()` matches that without discarding
samples.
- **Benchmarks that assume productive input** - aligners/annotators are usually
evaluated on productive reads.

Leave it off when you specifically want the unconstrained mix (including
non-productive rearrangements) - e.g. to study out-of-frame rates or to
stress-test a tool on non-productive input.

!!! warning "Immunoglobulin (BCR) only"
The productive contract's anchor checks assume BCR semantics, so it is an
immunoglobulin feature. TCR refdata accepts the call but raises a
`ValueError` at compile time - don't add `productive_only()` to a TCR
pipeline.

The constraint is implemented as a bundle of engine contracts (anchor
preservation, junction frame, no-junction-stop); the contributor-facing details
live in the [architecture guide](../architecture/index.md).
4 changes: 2 additions & 2 deletions site_docs/guides/recombination-editing.md
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,6 @@ not let you accidentally split them.
biology stage that runs *after* recombination edits; SHM
accumulates on the post-revision V and the inverted D.
- For the engine-side mechanics + audit details, see
[`docs/d_inversion_design.md`](https://github.com/MuteJester/GenAIRR/blob/master/docs/d_inversion_design.md)
[`audit-docs/d_inversion_design.md`](https://github.com/MuteJester/GenAIRR/blob/master/audit-docs/d_inversion_design.md)
and
[`docs/receptor_revision_design.md`](https://github.com/MuteJester/GenAIRR/blob/master/docs/receptor_revision_design.md).
[`audit-docs/receptor_revision_design.md`](https://github.com/MuteJester/GenAIRR/blob/master/audit-docs/receptor_revision_design.md).
6 changes: 3 additions & 3 deletions site_docs/guides/recombination-junction.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ walker can't disambiguate between alleles from the assembled
sequence alone:

```python
rec["v_call"] # 'IGHV1-2*02,IGHV1-2*04' ← tie set: two alleles equally consistent
rec["v_call"] # 'IGHVF1-G1*01,IGHVF1-G1*02' ← tie set: two alleles equally consistent
```

Two facts to know about call provenance:
Expand Down Expand Up @@ -125,11 +125,11 @@ the universe:
# Per-experiment subset:
ga.Experiment.on("human_igh") \
.recombine() \
.restrict_alleles(v=["IGHV1-2*02", "IGHV3-23*01"])
.restrict_alleles(v=["IGHVF1-G1*01", "IGHVF1-G2*01"])

# Per-cartridge weights:
cfg.reference_models = ReferenceEmpiricalModels(
allele_usage=AlleleUsageSpec(v={"IGHV1-2*02": 100.0, "IGHV3-23*01": 60.0}),
allele_usage=AlleleUsageSpec(v={"IGHVF1-G1*01": 100.0, "IGHVF1-G2*01": 60.0}),
)
```

Expand Down
Loading
Loading