Biofilter 4.2.0 — DuckDB + Parquet backend

4.2.0 (22/06/2026):

• New backend: read-only DuckDB over a Parquet bundle
  • New URI scheme parquet:///path/to/bundle/tables recognised by the
    Database class
  • Translates internally to duckdb:///:memory: plus auto-registration
    of one SQL VIEW per *.parquet file in the directory (skipping
    _chr_* partition children — the consolidated parents are queried)
  • Uses SQLAlchemy StaticPool so every session shares the in-memory
    catalog where the views live
  • DuckDB's natural rejection of writes against read_parquet views
    enforces read-only at the storage layer; the new Database.read_only
    flag exposes the same information at the application layer
• New dependencies: duckdb, duckdb-engine (read-only Parquet path)
• Existing PostgreSQL and SQLite backends are unchanged — the new mode
  is purely additive
• POC validation (logged in notebooks/Andre/ADR/0001-...):
  • 10,000 rsIDs annotated through variant_molecular_effects
    (1.79 billion rows) in 1.18s with peak memory 89 MB on local NVMe
  • Same workload on the LPC over GPFS: 15.52s — still well below any
    practical threshold
  • End-to-end CLI report run (annotation_master_variant for three
    rsIDs) completes in 1.22s
• Operational impact: HPC deployments no longer require a bundled
  PostgreSQL container, a 480 GB pgdata bind mount, or per-user copies.
  The bundle is shared, read-only, multi-user-safe.
• Documentation: ADR-001 (notebooks/Andre/ADR/0001-duckdb-parquet- strategy.md) describes the rationale, alternatives, and the SQLite
  migration attempts that motivated this change.

Biofilter 4.1.4

4.1.4 (02/06/2026):

• Extended variant_modeling output with 6 new columns
  • variant_1_input / variant_2_input — original input string(s) the user supplied (comma-joined when multiple inputs resolve to the same variant_id); usable as join key to merge results back into the caller's source table
  • variant_1_ref / variant_2_ref — reference allele from variant_masters
  • variant_1_alt / variant_2_alt — alternate allele from variant_masters
• variant_modeling lookup queries now select reference_allele and alternate_allele alongside chromosome and position
• Updated report_variant_modeling.md with new column descriptions and a "Merging results back to the caller's table" example using variant_*_input

Archived Versions

We are working on a new and improved version of Biofilter that incorporates the latest advancements. The versions linked here are for archival purposes only and should not be used.

Biofilter-2.4.3

We are working on a new and improved version of Biofilter that incorporates the latest advancements. The versions linked here are for archival purposes only and should not be used.

Python version 3.x